Digital Sesnory MArketing
Digital Sesnory MArketing
com
ScienceDirect
Journal of Interactive Marketing 45 (2019) 42 – 61
www.elsevier.com/locate/intmar
Abstract
People are increasingly purchasing (e.g., food, clothes) and consuming (e.g., movies, courses) online where, traditionally, the sensory
interaction has mostly been limited to visual, and to a lesser extent, auditory inputs. However, other sensory interfaces (e.g., including touch
screens, together with a range of virtual, and augmented solutions) are increasingly being made available to people to interact online. Moreover,
recent progress in the field of human–computer interaction means that online environments will likely engage more of the senses and become more
connected with offline environments in the coming years. This expansion will likely coincide with an increasing engagement with the consumer's
more emotional senses, namely touch/haptics, and possibly even olfaction. Forward-thinking marketers and researchers will therefore need to
appropriate the latest tools/technologies in order to deliver richer online experiences for tomorrow's consumers. This review is designed to help the
interested reader better understand what sensory marketing in a digital context can offer, thus hopefully opening the way for further research and
development in the area.
© 2019 Direct Marketing Educational Foundation, Inc. dba Marketing EDGE. All rights reserved.
Keywords: Consumer behavior; Digital marketing; Sensory marketing; HCI; Online environment
if one could actually taste the dishes that one sees on Instagram,
COLOURED, STEREOSCOPIC FEELY. WITH SYNCHRO- or feel the warmth of the virtual sand under your feet, not to
NIZED SCENTORGAN ACCOMPANIMENT. “Take hold of mention smell the coconut oil, when viewing your friends'
those metal knobs on the arms of your chair,” whispered travel photos on Facebook. And who would not want you to
Lenina. “Otherwise you won't get any of the feely effects.” virtually embrace their partner before saying goodbye after a
(Huxley 1932, p. 119) Skype call? The current lack of genuinely multisensory
Introduction interaction with the online environment is undoubtedly a
missed opportunity given that we are spending ever more of our
Who has not wondered, when browsing the website of an time on the Internet (Statista 2016).
online retailer, what one would look like wearing that new Digital interactive technologies (which enable the creation
sweater, or what it might feel like against the skin; or perhaps and/or manipulation of products on the screen), especially
whether those new Chinese noodles really would taste as good sensory-enabling technologies (i.e., SETs, those that can
as they look? Just imagine, for instance, how great it would be deliver sensory inputs), can be helpful when, for instance, it
comes to creating a “webmosphere” (i.e., the conscious
designing of web environments to create positive effects).
These technologies can also help inform the consumer about
⁎ Corresponding author at: Department of Marketing, Kedge Business
School, Domaine de Luminy, Rue Antoine Bourdelle, 13009 Marseille, France.
those other sensory properties of a product (e.g., its texture,
E-mail address: [email protected] (O. Petit). smell, and possibly even taste) that are simply not available
https://doi.org/10.1016/j.intmar.2018.07.004
1094-9968© 2019 Direct Marketing Educational Foundation, Inc. dba Marketing EDGE. All rights reserved.
O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61 43
currently in most (primarily visual) online environments marketing and those working in the field of human–computer
(Childers et al. 2001; Eroglu, Machleit, and Davis 2001; interaction (HCI). It is our belief that marketers need to better
Gallace et al. 2012; Hsieh et al. 2014; Kim and Forsythe 2008a; familiarize themselves with the full range of SETs that are
Rose et al. 2012; Song and Zinkhan 2008). SETs include both available while those working in HCI may benefit from
those devices that are already widespread (such as headphones making themselves aware of some of the potentially profitable
and touch screens), as well as a whole host of other new uses that their technologies might one day permit (Velasco et
technologies that have yet to be fully commercialized in this al. 2018). In order to achieve these goals, we return to the
context such as virtual reality (VR), augmented reality (AR), main sensory marketing theories that can be used to help
and even digital taste/smell interfaces. understand consumer behavior in the online environment. In
We believe that marketers can do a much better job when it the next sections, we explain, based on theories and practices,
comes to considering and integrating these various technologies how sensory information can be delivered more effectively
and their potential evolution to make the multisensory online online by means of digital interfaces that have evolved the
experience more engaging, immersive, informative, and, ulti- most in recent years (i.e., visual and haptic devices). We also
mately, enjoyable in the future. Doing so will likely help illustrate how the latest advances in other SETs (e.g., in
companies to differentiate themselves from the competition in the auditory, olfactory, and gustatory devices) can potentially be
crowded online marketplace. Future research is therefore really used to reinforce this communication and even suggest new
needed in order to better understand how SETs can be used to multisensory marketing strategies. We finish by providing
enhance the consumer experience (e.g., how immersed they are in some ideas concerning potentially fruitful directions for
“the experience” and how persuasive such experiences are) and further research (see Table 1 for a summary of the
nudge the latter's behaviors (e.g., how much do they choose to technology).
spend, and on what?).
The main objective of this article is therefore to introduce a Theoretical Framework: What Does Sensory Marketing
new way of thinking about (digital-) sensory marketing. This Mean in the Online Environment?
new approach focuses on the use of digital technologies in
online contexts, based on theories and concepts taken directly According to the theory of embodied cognition, all cognitive
from the growing field of sensory marketing research. We try processes are grounded in bodily states and in the brain's
to bridge the gap between those researchers in sensory sensory modality-specific processing systems (Barsalou 2008;
Table 1
Summary of common and new sensory-enabling technologies
Common interfaces New sensory-enabling technologies
Sense Means/cues Concepts Means/cues Concepts
Sight Screen: Font, icon, Mental imagery (Cian et al. 2014; Eelen et al. 3D-interactive view,
Mental imagery (Choi and Taylor 2014; Huang and Liao
picture, videos (color 2013; Elder and Krishna 2012; Petit et al. virtual try-ons,
2017), telepresence/ immersion (Animesh et al. 2011;
depth, size, position, 2016) augmented reality.
Klein 2003; Li et al. 2002; Nah et al. 2011; Yim et al.
dynamic). sensory congruency (Sunaga et al. 2016; 2017), enjoyment (Kim and Forsythe 2008a, b; Lee and
Velasco et al. 2015; Velasco et al. 2016b; Chung 2008; Nah et al. 2011; Yim et al. 2017), flow
Woods and Spence 2016), interactivity (Song (Animesh et al. 2011; Huang 2012; Huang and Liao
and Zinkhan 2008; Van Noort, Voorveld, and 2017; Jiang and Benbasat 2004; Nah et al. 2011; Novak,
Van Reijmersdal 2012). et al. 2000; Van Noort, Voorveld, and Van Reijmersdal
2012), interactivity (Huang 2012; Yim et al. 2017); self-
congruity (Merle et al. 2012), ownership (Brengman et
al. 2018; Huang and Liao 2017), need for touch
(Brengman et al. 2018; Choi and Taylor 2014),
curiosity (Beck and Dominique Crié 2018).
Hearing Headphones, loud- Sensory congruency (Hagtvedt and Adam Multisensory Sensory congruency (Hashimoto et al. 2008; Ho et al.
speaker (music, Brasel 2016; Knoeferle et al. 2016). experience with 2013; Liu, Hannum, and Simons 2018)
sound, jingle). auditory inputs (Food
simulator, Straw-like
User Interface).
Touch Mouse, touchscreen. Mental imagery (Shen et al. 2016), Vibrotactile interfaces, Need for touch (Brasel and Gips 2014; Cano et al. 2017;
ownership (Brasel and Gips 2014), affect body-grounded tactile Jin 2011), telepresence (Leithinger et al. 2014; Sallnäs,
(Brasel and Gips 2015; Shen et al. 2016) actuators, mid-air Rassmus-Gröhn, and Sjöström 2000), emotion (Rantala
haptics. et al. 2013), Midas touch effect (Haans and IJsselsteijn
2009; Haans et al. 2014; Spapé et al. 2015).
Smell X X Multisensory Sensory congruency (Ranasinghe et al. 2018; Liu,
experience with smell Hannum, and Simons 2018).
inputs (Season
Traveler,
MetaCookie+).
44 O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61
Niedenthal et al. 2005). Thus, all consumer experiences are reinforce perceptual reenactments, notably, by improving the
based on the integration of sensory inputs that affect the latter's feeling of immersion.
judgment and behavior (Krishna 2012). Therefore, by engaging In addition to images, devices such as computers and
the consumers' senses more effectively, sensory marketing smartphones can facilitate auditory (via loudspeakers) and
strategies can potentially impact the decision-making process in haptic interactions (via touch screens and vibrations) with a
the store (Krishna 2012; Krishna and Aydınoğlu 2017; Spence positive effect on product evaluation. These sensory inputs
2012; see Spence et al. 2014, for a review). However, one might also elicit perceptual re-enactments in other sensory
might wonder what impact sensory marketing can really have modalities. For example, Kitagawa and Igarashi (2005) used
during online shopping when interactions with the environment sound to induce virtual touch sensations. They gave the
are limited to a computer screen. impression to their participants that their ears had been tickled
According to the theory of embodied cognition, cognition by presenting the sound of a brush stroking the ear of a dummy
can be situated (i.e., operated directly on the real-world head. Several studies have also shown that hearing and touch
environment, i.e., online embodiment), or decoupled from the can be used to stimulate visual imagery (de Volder et al. 2001;
real-world environment (offline embodiment, Wilson 2002). A Lacey et al. 2010). Although product pictures are generally
priori, the options for communicating sensory information in present on websites, by broadcasting sounds, producing
the online environment would appear to be rather limited. After vibrations, and allowing consumers to zoom/rotate the images
all, it has traditionally not been possible to touch, smell, or taste with their fingers, marketers might give them a better visual
objects over the Internet (see Gallace and Spence 2014, for a representation. Moreover, these sensory inputs might facilitate
review). Thus, the online environment might be considered as a multisensory integration, and thus have a positive effect on
context of offline embodiment, in which interactions with the visual attention and search (Spence 2011).
world occur only through digital interfaces. However, this does Recent progress in HCI has also led to the suggestion that
not mean that the senses stop affecting cognition in the online marketers might be able to benefit from new multisensory tools
environment. In this context, cognitive activity is still supported (including olfactory and even taste stimulation, Obrist et al.
by modality-specific sensory systems (Niedenthal et al. 2005). 2016; Petit et al. 2015; Spence et al. 2017). Thus, it might not
When consumers experience stimuli in the real-world (e.g., be necessary to go via mental imagery in order to fill in the
eating potato chips), the brain captures perceptual, motor, and missing sensations in the online environment. The SETs offer a
introspective states relating to the various senses and integrates glimpse of a new “online” embodied environment, providing
them into multisensory representations that are stored in multisensory experiences similar to those observed in the real
memory (e.g., texture of the chips crunching between the world. In the following sections, we show that the visual-
teeth, Barsalou 2008; Papies and Barsalou 2015). Later, the enabling technologies are now able to improve perceptual re-
exposure to product pictures (potato chips) in online stores can enactments in the online environment with positive effects on
trigger spontaneous perceptual re-enactments (i.e., embodied both consumer experience and product evaluation. Thereafter,
mental simulations: think of this as a more automatic form of we highlight how the other SETs (i.e., already haptic and
mental imagery) of those multisensory representations (Chen, auditory, and eventually potentially even olfactory and
Papies, and Barsalou 2016; Petit et al. 2016). These perceptual gustatory devices) are likely to improve the effects of visual
re-enactments engage some of the same brain areas that were devices, while suggesting new forms of interaction with the
recruited during the previous experiences which, in turn, can consumer online (see Table 2 for a summary of the research).
produce similar sensations (Simmons, Martin, and Barsalou
2005; Okajima et al. 2016). Visual-Enabling Technologies: A New Form of Mental
Perceptual re-enactments have been observed in different Imagery
senses. Seeing the picture of a given food or reading its name
can activate the olfactory and the gustatory cortices (González Icons, pictures, font, and videos all constitute visual stimuli
et al. 2006; Simmons, Martin, and Barsalou 2005). Similarly, (and are key elements associated with a brand) that marketers
the sight of lip movements appears to stimulate the auditory can adjust (in terms of the resolution, color, depth, size,
cortex too (Calvert et al. 1997). A recent study also showed that position, etc.) over the Internet in order to improve consumer
watching the hand of someone else grasping food leads to experience. Moreover, it is currently possible to include 3D
activations in motor-related brain areas (Basso et al. 2018). objects (see Fig. 1a, Algharabat et al. 2017) as well as to create
These studies suggest that through perceptual re-enactments, VR environments (see Fig. 1b; Jin 2009). Visual-enabling
the consumer's senses might be stimulated online. More technologies include larger views (super close-up; zoom in/out;
specifically, the perceptual re-enactments produced by images enlargement), alternate views (e.g., views from 2 to 3 angles),
on websites can serve to fill in the missing features of the 3D-interactive view (views from every angle as a consumer
products that are not physically present (this can be thought of drags their mouse), and virtual try-ons (VTO). In addition, they
as perceptual completion, Pessoa and De Weerd 2003; Spence allow a consumer to zoom in on the product that they happen to
and Deroy 2013). Thus, by viewing product-related images on be interested in, to rotate it and, by so doing, view it from a
websites, consumers might define sensory expectations, and variety of different angles (Kim and Forsythe 2008a). In
even offset their need for touch. We will see in the following addition, they can undoubtedly change the way in which the
sessions that the new visual-enabling technologies might help consumer interacts with content online. They might be used to
O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61 45
Table 2
Research summary on digital sensory marketing concepts and interfaces.
Sensory design Relevant literature Key findings
examples
Virtual experience 3D virtual Animesh et al. (2011) Interactivity which the environment has a positive impact on telepresence and flow.
environment Gabisch (2011) Self-image congruence and perceived diagnosticity moderate the effects of virtual world on
purchase intent.
Huang (2012) Affective involvement has a more positive effect than flow on purchase intention in virtual
world.
Jin (2009) Modality richness and prior involvement positively impact shopping behavior in 3D virtual
stores.
Lee & Chung (2008) Virtual shopping mall creates stronger quality assurance and enjoyment than ordinary mall.
Nah et al. (2011) Virtual world increases telepresence and enjoyment with a positive effect on brand equity.
AR environment Beck and Dominique Crié Using a Virtual Fitting Room on a website increases curiosity about the product, intention
2018 to patronize (online and offline) and intention to purchase (online and offline).
Poushneh & AR has a positive effect on user experience that subsequently influences satisfaction and
Vasquez-Parraga 2017 willingness to buy.
Yim et al. (2017) AR has a positive influence on novelty, immersion, enjoyment, and usefulness, resulting in
positive attitudes and purchase intentions.
Scholz & Duffy (2018) AR shopping app used at homes creates a close and intimate (rather than transactional)
relationship with the brand.
Force feedback Iwata et al. (2004) Users experience food textures by using a device generating a force on their teeth.
Audio-tactile Hashimoto et al. (2008) Users virtually experience the sensations of drinking with straw trough pressure change in
interface the mouth, vibrations on the lips, and sound.
Digital taste Ranasinghe et al. (2017) The sensor produces electrical stimulations on the user’s tongue with the hope of
interface manipulating the sourness.
Product evaluation Visual features Christian et al. (2016) 3rd (vs. 1st) person perspective decreased the mental representation, actual consumption,
and willingness to pay for unhealthy food.
Cian et al. (2014) Perceived movement evoked by pictures stimulate dynamic imagery that positively affects
consumer engagement.
Eelen et al. (2013) Monitoring orientation cues affects product evaluation and choice.
Elder & Krishna (2012) Product orientation (handle leftwards vs. rightwards) affect purchase intent.
Gvili et al. (2017) Evoked motion in food pictures enhance projected taste and freshness.
Shen & Sengupta (2012) Occupying the dominant (vs. non-dominant) hand impairs the ease of simulation which
leads consumers to lower evaluations of the product.
3D product Jiang & Benbasat (2004) Virtual product control has a positive impact on perceived diagnosticity and flow.
visualization Li et al. (2002) Virtual product experience increases telepresence with a positive impact on purchase intent.
Park et al. (2008) Rotation in online product presentation impacts perceived information quantity and mood
with a positive effect on attitude, and purchase intent.
Virtual try-on Cho & Schwarz (2012) The quality of image used to construct a virtual mirror play an important role in product
evaluation.
Huang & Lio (2017) Haptic imagery and sense of self-location during virtual try-on positively impact flow
experience.
Kim & Forsythe (2008a, b) Virtual try-on reduces product risk and increases the entertainment value of the online
shopping process.
Merle et al. (2012) Personalized (vs. non-personalized) virtual try-on elicits higher utilitarian value and
purchase intent.
Touch screen Brasel & Gips (2015) Touch screen (vs. mouse interface) increases the number of alternatives searched, and leads
the consumer to more tangible attributes and internal sources of information in the choice
process.
Shen et al. (2016) Touch screen (vs. mouse interface) enhances the choice of a hedonic option over a
utilitarian one.
Auditory features Ho et al. (2013) Sound feedback (vs. no sound) from material products during virtual trial increase the
willingness to pay.
Mid-air haptic Obrist et al. (2013) Ultrasound waves with modulation frequencies (16Hz, 250Hz) create textiles and wind/
breeze sensations.
Leithinger et al. (2014) Haptic interface allowing the user to manipulate objects remotely.
(continued on next page)
46 O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61
Table 2 (continued)
Sensory design Relevant literature Key findings
examples
Shapeshifting
display
Need for touch 3D product Choi & Taylor (2014) For geometric products: 3D product image have higher persuasive effects than 2D product
visualization image for both high- and low-NFT consumers. For material products, 3D visualization only
have a positive effect for low-NFT consumers. Mental imagery mediates the persuasive
effects of the 3D versus 2D format.
Touch screen Brasel & Gips (2014) Touch screen (vs. mouse interface) elicits stronger feelings of perceived product ownership,
with stronger effects for material products.
Augmented reality Brengman et al. (2018) AR product manipulation results in higher levels of perceived ownership, with stronger
effect for material products.
Force feedback Jin (2011) Force feedback (vs. no force feedback) elicits more positive product evaluation, test-driving
experience, and brand-self connection for consumers high in instrumental NFT.
Tactile features Cano et al. (2017) Product rotation and scrunch increase use engagement for material products, regardless
their NFT.
Midas touch effect Force feedback Sällnas (2000) Haptic force feedback increases perceived social presence.
Vibrotactile Haans & IJsselsteijn Vibrotactile touch (vs. no touch) does not provide more helping behaviour.
feedback (2009)
Haans et al. (2014) Helping behaviour was higher in the vibrotacile (vs. no) touch condition when participants
who initiated the virtual touch knew the purpose of the study in advance.
Rantala et al. (2013) Squeeze is better to communicate unpleasant and aroused emotion, while finger touch is
better for pleasant and relaxed emotion.
Spapé et al. (2015) Vibrotactile feedback affects generosity (increasing an offer) but not direct compliance
(accepting an offer).
Human-shaped Sumioka et al. (2013) Conversations with a remote partner using huggable human-shaped device (vs. mobile
cushion phone) reduces the cortisol levels (stress hormone).
Sensory congruency Visual features Sunaga et al. (2016) Visual search is facilitated when light (dark) coloured products are positioned in the upper
(lower) shelf positions.
Velasco et al. (2015) Semantic congruence between colour (e.g., red) and flavour (e.g., tomato) facilitates visual
search.
Velasco et al. (2016b.) Rounder designs are evaluated more often as sweeter than angular designs.
Woods & Spence (2016) Specific colours (e.g., red, green, black, and white) can help to communicate basic tastes
(e.g., sweet, sour, bitter, and salty).
Auditory features Hagtvedt and Low (vs. high) sound frequency leads people to fixate on dark (vs. light) objects faster and
Brasel 2016 longer and increase purchase intent.
Knoeferle et al. Semantic congruence between sound (e.g., popping sound) and product (e.g., bottle of
(2016) Champagne) facilitates visual search
Spence & Zampini (2007) Aerosol sprays are perceived as being more pleasant (but significantly less forceful) when
the high-frequency sounds is attenuated.
Zampini & Spence (2004) Potato chips are perceived as being both crisper and fresher when the high frequency
facilitate consumers' perceptual re-enactments, and help them visual-enabling technologies have been used to improve web
to fill in the missing sensory inputs (Spence and Deroy 2013). atmospheres. They may create a sensation of immersion (or
In the next subsections, we discuss how these new forms of telepresence), detaching people from the physical reality, thus
interaction through visual-enabling are likely (or not) to absorbing them in their virtual experience (Animesh et al. 2011;
improve the experience and product evaluation. Klein 2003; Li, Daugherty, and Biocca 2002; Nah,
Eschenbrenner, and Dewester 2011; Yim, Chu, and Sauer
Embodied Online Experience 2017). It has been argued that 3D environments might deliver
higher levels of enjoyment than 2D or physical environments
Much of the research has demonstrated that carefully (Kim and Forsythe 2008b; Lee and Chung 2008; Nah et al.
considered visual features can make the online experience 2011). Crucially, this seemingly more enjoyable experience
more immersive, aesthetically pleasing, and enjoyable (Bölte et provided by virtual and augmented reality was found to have a
al. 2017; Childers et al. 2001; Eroglu et al. 2001; Rose et al. positive impact on both purchase intentions and on the
2012; Varadarajan et al. 2010). Li, Daugherty, and Biocca consumers' willingness to pay (Animesh et al. 2011; Beck
(2001, p. 14) developed the concept of virtual experience to and Crié 2018; Gabisch 2011; Jin 2009; Poushneh and
represent “psychological and emotional states that consumers Vasquez-Parraga 2017). Thus, the virtual environment seems
undergo while interacting with products in a 3-D environ- to provide a fuller experience to the consumer. From the point
ment”. According to a recent review by Javornik (2016a), of view of the theory of embodied cognition, this would lead to
O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61 47
Fig. 1. (a.) 3D product presentation, Algharabat et al. (2017); (b.) Spokes-avatars in a virtual retail store, Jin (2009); (c.) Personalized virtual try-on: Augmented-
reality interactive technology, Huang and Lio (2017).
considering the online environment as a place of online Embodied Online Product Evaluation
embodiment, in which the perceptual, motor, and introspective
states across the various senses should be considered in detail As noted earlier, seeing a picture of an object reactivates, at
(Barsalou 2008; Niedenthal et al. 2005). least in part, the same brain areas that were mobilized during the
It should, however, be noted that to date the research has previous perceptual episodes (Barsalou 2008; Simmons, Martin,
mainly highlighted the impact of visual-enabling technologies and Barsalou 2005). Thus, simply by displaying a picture of a
on the affective reactions to the online experience (see Javornik food on the screen, the wily marketer may be able to stimulate
2016a). Their effects on the flow, a cognitive state experienced mental images of its texture, smell, and even flavor that can
during online navigation, in which consumers are completely facilitate the customer's evaluation of the food (Elder and
absorbed in their activity, are rather mixed (Animesh et al. Krishna 2012; Krishna, Cian, and Sokolova 2016; Petit et al.
2011; Huang 2012, Huang and Liao 2017; Jiang and Benbasat 2017; Spence and Deroy 2013; Okajima et al. 2016). In order to
2004; Novak, Hoffman, and Yung 2000; Van Noort, Voorveld, help consumers mentally simulate interactions with products
and Van Reijmersdal 2012). These mixed effects on the during their online experiences, marketers can change the way in
consumer's cognitive state is all the more problematic because which the product image is presented (Krishna and Schwarz
that utilitarian value (i.e., functional benefit) is more strongly 2014). For example, they may want to favor the use of dynamic
related to preference toward the Internet retailer and online (over static) images of the product (Cian, Krishna, and Elder
buying than hedonic value (i.e., experiential benefits, Bridges 2014; Gvili et al. 2017; see Okajima et al. 2016, for a review).
and Florsheim 2008; Overby and Lee 2006). However, this By using dynamic images, marketers can increase the
does not necessarily mean that the impact of sensory marketing consumer's ability to generate mental simulations of transfor-
on consumer behavior is limited in the online environment, but mation, rotation, and reorganizations of the imagined product
that it does not necessarily go through the same channels as with a positive effect on its evaluation (Cian, Krishna, and
those used in the physical environment. In the next subsection, Elder 2014). At the cerebral level, Basso et al. (2018) found that
we show how visual-enabling technologies are likely to affect watching videos featuring a hand grasping food (vs. object)
online behavior by proposing new ways to interact with the leads to an increase of activity in somatosensory-motor brain
product. areas. Neural activity in these areas is often seen when grasping
48 O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61
objects and could potentially facilitate the simulation of food consumer (Cho and Schwarz 2012; Javornik 2016b; Kim and
consumption (Chen et al. 2016; Vingerhoets 2014). Similar to Forsythe 2008b; Merle, Senecal, and St-Onge 2012). Cho and
dynamic images, mental simulations might also be stimulated Schwartz highlighted that VTO positively impacts people's
simply by orienting a product on the screen in the direction of product evaluation, especially when the latter upload their
the hand that is normally used when grasping (e.g., Eelen, favorite (rather than just a conveniently available) pictures of
Dewitte, and Warlop 2013; Elder and Krishna 2012; Shen and themselves. Similarly, Merle, Senecal, and St-Onge 2012
Sengupta 2012), or by changing the perspective from which a reported that personalized (vs. non-personalized) VTO im-
product is viewed (Basso et al. 2018; Christian et al. 2016). proves self-congruity and the confidence of consumers in their
Therefore, one might expect that simply by capitalizing on such product choices. More recently, Huang and Liao (2017) also
visual manipulations, the marketer could make a customer's demonstrated that VTO can have a positive impact on flow
online product evaluation more immersive, despite the experience, by affecting perceived ownership, and self-
separation necessarily created by viewing a product on a screen. explorative engagement. Therefore, VTO might help reduce
The new visual-enabling technologies are likely to enhance “bracketing,” a trend to buy multiple products, select the best
this sensation of immersion. 3D images give the user the one, and then returning the rest (Sharma 2017). However, while
feeling of being able to interact with the product itself, and these new visual-enabling technologies offer a more direct
thus stimulate mental simulations of product interaction (Li, interaction with the product than 2D visuals, it is not certain
Daugherty, and Biocca 2001). Such effects of 3D images on that this interaction is sufficient to allow consumers to evaluate
product evaluation can be reinforced by allowing the user to all the products. Some products may have material properties
spin/rotate the product, thus enabling them to examine the difficult to evaluate without a real touch, this point is discussed
product from all possible angles. For instance, in one study, in the next subsection.
Park, Stoel, and Lennon (2008) displayed two pairs of khaki
trousers (rotating vs. non-rotating) on a website. They found Need for Touch for Visual Evaluation
that the rotating version increased perceived information
quantity among the participants with a knock-on positive Researchers have demonstrated that some people feel a need
effect on attitude and purchase intentions. to touch the product (or imagine touching it) in order to be
Jai, O'Boyle, and Fang (2014) analyzed the effect of format confident in their choices (Peck, Barger, and Webb 2013; Peck,
(static, zooming and rotate views) of pictures of dresses on and Childers 2003a, b; Peck and Shu 2009; Shu and Peck 2011).
brain activity during the encoding (visual presentation period) However, the need for touch (NFT) varies depending on the
and decision processes. During the decision process, women tactile properties of the products themselves. Some products, for
were instructed to create a mental image of the product and to instance, have more salient geometric (e.g., shape, size, and
indicate how much money they wanted to bid for the dress. structure) and other material (e.g., texture, temperature, and
They found that during the encoding period, image zooming weight) properties than do others (see Choi and Taylor 2014;
(vs. static images) led to higher activations in the primary visual Lederman 1974; Spence and Gallace 2008). An image and/or a
cortex, but did not lead to higher modulation in motor areas. As written description of the haptic properties might therefore be
such, these results suggest that image zooming provides more sufficient to evaluate a product with more salient geometric
detailed visual information than static images, but does not properties (e.g., a smartphone). However, consumers might need
stimulate mental simulations of grasping movements. They also to touch a product with more salient material properties (e.g., a
failed to find any difference between both images during the sweater) in order to evaluate it, especially those consumers with
decision process. By contrast, during this period, rotation a high NFT (McCabe and Nowlis 2003; Peck and Childers
videos (vs. static images) led to higher activities in visual and 2003a, b). Thus, some haptic dimensions might be easier to
premotor cortices, suggesting that participants had more vivid simulate than others in an online environment.
mental images of product interactions in their mind. In this Visual-enabling technologies have been shown to enhance
condition, participants also present more activity in those areas the ability of consumers to imagine touching and trying-on
known to code the reward value of stimuli (caudate nucleus and products on a shopping website (Li, Daugherty, and Biocca
putamen), and self-related mental processing (precuneus) areas 2001; 2002). Thus, these technologies appear interesting as far as
(Fransson and Marrelec 2008; Knutson et al. 2007). These addressing the NFT of consumers in virtual environments is
results suggest that rotation view might promote a better sense concerned (Peck and Childers 2003a, b). For instance, Choi and
of self-referencing with a higher level of product preference. Taylor (2014) demonstrated that 3D images can stimulate mental
Self-referencing can also be improved by providing a more imagery with a knock-on positive impact on persuasion. In their
personalized virtual experience. VTO allows the shopper to use study, websites with 2D and 3D formats were developed in order
an avatar, create their own virtual models based on their facial to advertise two products with different haptic properties: a
characteristics, or else even to use a “virtual mirror” (created watch (geometric properties) and a jacket (material properties).
with their own digital photo uploaded to a retailer's Website, The results indicated that 3D advertising for the watch created
Pantano and Naccarato 2010; or by using augmented-reality more vivid mental images with a positive effect on attitude
interactive technology, see Fig. 1c, Huang and Liao 2017). The toward the brand, purchase intentions, and intention to revisit the
research suggests that VTO can provide reliable information website as compared to the 2D advertising. However, for the
regarding the fit and how a product might look on the potential jacket, 3D advertising only exerted a positive effect on product
O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61 49
evaluation for those people with a lower NFT. Thus, consumers online (see Brasel and Gips 2015; Shen, Zhang, and Krishna
with a high NFT might need to physically contact material 2016). For instance, the participants in one study by Brasel and
products online in order to facilitate their decision-making. This Gips had to search for hotels on a travel review website using
physical contact could be provided by the new haptic-enabling either a touch screen or a mouse interface. Interestingly, those
technologies, as discussed below. who used the touch screen mentioned more tangible elements
of the room (e.g., referring to its décor and furniture), and
Haptic-Enabling Technologies: A New Form of Physical considered internal sources of information (e.g., gut feel and
Contact instinct) as being more important in their choice process. By
contrast, those who used the mouse were more affected by
While consumers cannot literally touch the products that external objective sources (user reviews and star ratings), and
they see online, they normally do interact haptically with mentioned more intangible attributes as instrumental in making
multiple interfaces already (e.g., mice and touch screens). their decision (e.g., the availability of Wi-Fi and employee
These haptic interfaces might compensate for the lack of actual demeanor). Thus, touch-screen devices would appear more
touch in those who feel a high NFT. Moreover, several devices likely to bias the online purchase process than more traditional
have already been developed (though are still not widespread) mouse interfaces.
to improve haptic interactions through the Internet, such as Using a haptic interface can also affect the consumer's
vibrotactile interfaces (Kim et al. 2013), body-grounded tactile preference. Shen, Zhang, and Krishna 2016 found that using a
actuators (tapping on, squeezing, and twisting, Stanley and direct touch interface compared with a non-touch interface
Kuchenbecker 2011), or even mid-air haptics (Ablart, Velasco, made people more likely to choose a hedonic option over a
and Obrist 2017; Obrist, Seah, and Subramanian 2013; Vi et al. utilitarian one. They also demonstrated that mental interaction
2017; see Huisman 2017, for a recent review). These with the products mediates this direct-touch effect. These
technological developments may prove useful as far as results suggest that similar to visual design, haptic design can
improving physical interactions with both objects and people be manipulated to enhance mental simulations of product
are concerned (Brengman, Willems, and Van Kerrebroeck interaction. Based on the available evidence, marketers should,
2018; Chung, Kramer, and Wong 2018; Kerrebroeck, Helena, whenever possible, therefore consider whether it is possible to
and Brengman 2017). Below, we describe how such develop- use different interfaces as a function of the kind of product that
ments may impact product selection, the NFT, and even they wish to promote.
interpersonal interactions.
Need for Touch for Material Evaluation
Haptic Product Evaluation
Visual-enabling technologies have proven unsuitable for the
Using a direct touch interface has consequences for how evaluation of products with material properties by individuals
people search for a product or service and make their choices with a high NFT (Choi and Taylor 2014). However, the online
environment offers the consumer other ways in which to In the future, marketers will likely also want to promote
interact with the products that they may happen to be interpersonal relationships in the online environment in order to
evaluating. For instance, Brasel and Gips (2014) demonstrated make their website appear more trustworthy, or even promote
that touch screens elicit stronger feelings of perceived product word of mouth on social media. Kreijns et al. (2007) suggest
ownership than touchpads or mice. In turn, this perceived that improving the perceived sociability of a website is likely to
ownership increases what people are willing to pay in order to facilitate a consumer's trust, belongingness, and sense of
acquire the products. They also found that the touch– community. Animesh et al. (2011) also highlighted that
ownership link is stronger for those products that have high perceived sociability in the virtual environment is positively
material properties (e.g., a sweatshirt as compared to a city related to the experience of flow. Haptic-enabling technologies
tour). By providing a more direct interaction with the product might help to improve interpersonal relationships when
than visual interfaces, haptic interfaces have no doubt communication is over the Internet. Touch is very important
stimulated mental imagery and thus increased the perception when it comes to establishing secure attachments and
of ownership for material products. However, these techniques interpersonal connections between people (see Gallace and
have not been successful in compensating for the NFT of Spence 2014, for a review; Guerrero and Andersen 1994).
material products, something that might potentially be Touching someone can result in prosocial behavior, a
addressed by certain of the new haptic interfaces (Cano et al. phenomenon known as the “Midas touch effect” (Crusco and
2017; Jin 2011). For example, Cano et al. used a digital tool Wetzel 1984). For instance, Crusco and Wetzel found that
called Shoogleit which allows the user to virtually pinch and when a server physically touches a customer, it leads to an
scrunch a section of the clothing fabric with their fingertips on increase in the size of the tip. Similarly, being touched by a
a tablet during product evaluation (see Fig. 2). This salesperson can give rise to a feeling of social attachment,
technology contributed to a higher level of user engagement, which can then enhance the evaluation of products and services
regardless of participants' NFT. Thus, further studies should be (Hornik 1992). Social touch also has the power to communicate
O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61 51
specific emotions. So, for example, Hertenstein et al. (2006) Multisensory-Enabling Technologies: The Future of the
showed that hitting, squeezing, or shaking the forearm of Internet
another person can be used to communicate anger, whereas
love is mostly communicated by stroking, finger interlocking, The majority of life's most enjoyable experiences are
and rubbing. inherently multisensory (Spence 2002). In the real world, the
Similar to real touch, haptic devices allow people to induce more store atmospherics are multisensorially congruent, the
affective reactions over the Internet (Sumioka et al. 2013, see more pleasant and interesting they will likely be evaluated
Gallace and Spence 2014; Huisman 2017, for reviews). For (Mattila and Wirtz 2001; see Spence et al. 2014, for a review).
example, by producing specific patterns of vibrotactile The same is likely to be true for online environments (Dinh et
feedback, Rantala et al. (2013) were able to induce emotions al. 1999; Feng, Dey, and Lindeman 2016; Liu, Hannum, and
in users. Thus, unpleasant and high-arousal emotions were Simons 2018; Obrist et al. 2016; Spence et al. 2017). Previous
found to be better transmitted by means of squeeze-like research has shown that multisensory integration increases the
gestures, whereas the finger-touch gesture was more suitable likelihood that the brain detects a stimulus and/or initiates a
for pleasant and low aroused emotions. Other studies have also response to this stimulus (see Stein and Stanford 2008 for a
showed that digital haptic input can be used to increase feelings review). Furthermore, multisensory integration (and other
of telepresence (Sallnäs, Rassmus-Gröhn, and Sjöström 2000; forms of crossmodal interaction) might be facilitated by
see Gallace et al. 2012, for a review). Interestingly, though, the semantic congruency and cross-modal correspondences (Chen
Midas touch effect has proved difficult to reproduce using and Spence 2018; Spence 2011).
haptic-enabling technologies. For instance, Haans and Semantic congruency refers to those situations in which
IJsselsteijn (2009) failed to find a virtual “Midas touch effect” pairs of stimuli in different sensory modalities share common
when people were touched by a haptic device. More recently, identity or meaning (e.g., woofing sound paired with a static
though, Haans, de Bruijn, and IJsselsteijn (2014) succeeded in picture of a dog). Cross-modal correspondences describe a
demonstrating a virtual Midas touch effect, but only when the more general tendency for a feature, or attribute (e.g., larger/
confederate (i.e., the person who initiated the virtual touch) smaller objects), in one sensory modality to be matched (or
knew in which experimental condition (virtual touch vs. no associated) with a sensory feature, or attribute (e.g., lower/
touch) was the participant who had to exhibit signs of prosocial higher-pitched sounds), in another (Spence 2011). Based on
behavior. They may have been biased to elicit helping behavior cross-modal correspondences, mental imagery occurring in one
in the touch condition. Thus, it is possible that only specific sensory modality (not only visual) might result from the
prosocial behaviors might be affected by the virtual “Midas presentation of a physical stimulus in another. Cross-modal
touch effect”. Consistent with such a view, Spapé et al. (2015) mental imagery has been considered as a form of perceptual
reported that vibrotactile feedback affected generosity (i.e., completion and might thus be used to fill in the missing features
increasing the size of an offer) but not direct compliance (i.e., through the Internet (Spence and Deroy 2013).
accepting an offer). Cross-modal correspondences and semantic congruency
Without going as far as prosocial behavior, these technol- have been shown to influence performance across a range of
ogies can simply improve the felt closeness between people different tasks (e.g., speed of detection, perceptual discrimina-
(and between people and brands). For instance, Mueller et al. tion) that can be relevant to make decisions in the online
(2005) developed a device that allows one person to send a hug environment (Spence 2011). Specifically, in the following
to another by rubbing the belly of a stuffed animal. Similarly, subsections, we show how visual and auditory designs can be
the inFORM display allows the users to touch their hands used/combined in order to improve information search and
remotely, to extend the physical embodiment in the online sensory expectations, and how new multisensory-enabling
environment (see Fig. 3, Leithinger et al. 2014). Developing technologies can lead marketers to rethink consumer's online
this kind of interaction for use while on the Internet may be of experience.
interest to marketers, given that online consumer socialization
through peer communication plays a key role in purchase Sensory Congruency in Product Search
decisions (Wang, Yu, and Wei 2012). However, to facilitate
haptic communication online, cheaper and more convenient During online purchases on a retailer's websites (e.g., shoes,
interfaces, adaptable to computers and mobile phones, will digital cameras, or food), it is often necessary to display a large
likely need to be developed. For example, Park and Nam number of images representing relatively similar products. The
(2013) created a device to share haptic “pokes” during phone choice between items can be difficult to make. In this context,
calls. Pokes are sent through an inflatable surface on the front brands have to attract customers' attention on their product
of a mobile phone, while another person receives finger images in order to increase their chances of selection (Armel,
pressure inputs on the back of another phone. It is easy to Beaumel, and Rangel 2008; Milosavljevic et al. 2012). In order
imagine how such a device might one day be used on social to improve product saliency, marketers might want to ensure
media, to poke friends, followers, and even potential customers. that the visual features used to promote the products are
However, it is worth noting that such a Poke might still be very congruent with their other sensory attributes.
similar to a notification signaled by vibration of the mobile Sunaga, Park, and Spence (2016) highlighted that lighter
phone that can be turned off. colored objects tend to be perceived as lighter (in weight), and
52 O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61
objects appear lighter when they are presented at the upper part Sensory Congruency in Product Evaluation
of the visual field. Based on these associations, they were able
to facilitate their participants' visual search by using a display Once the customers' attention has been captured, they will
on the screen with light (dark) colored products positioned in need to analyze the attributes of the product to know whether or
the upper (lower) shelf positions. Here, the cross-modal not it meets their expectations (Dawar and Parker 1994).
correspondence between the lightness of product colors and Similar to visual search, visual and auditory features can
their location increased the speed and accuracy with which communicate sensory expectations by considering semantic
products were detected, facilitating visual search and product congruency and cross-modal correspondences. For instance,
selection. Similar correspondences also exist between light round shapes (e.g., logos, labels, figures, typefaces) have been
(dark) color and high (low) sound frequency (Marks 1987). shown to be more appropriate when it comes to communicating
Relevant here, Hagtvedt and Brasel (2016) recently found that sweetness, while bitter, salty, and sour tastes might be better
low (vs. high) sound frequency leads people to fixate on dark promoted through the use of more angular shapes instead (see
(vs. light) objects faster and for longer. Velasco et al. 2016b, for a review). Similarly, pink, white,
Semantic congruency can also affect visual search. For green, and black foreground colors should be used to enhance
example, Velasco et al. (2015) reported that people search for, people's expectations that a product is going to taste sweet,
and find, target products in online displays significantly faster salty, sour, and bitter, respectively (see Favre and November
when the color of the packaging happens to be (semantically) 1979; Wan et al., 2015; Woods and Spence 2016).
congruent with the flavor of the product (e.g., red/tomato) than At this stage, auditory features can be used to convey, or
when it is less congruent (e.g., yellow/tomato). Similarly, accentuate, the sensory features of a product online through
Knoeferle et al. (2016) demonstrated that using sounds that are semantic congruency (e.g., the crack of the chocolate of an ice-
semantically associated with particular brands/product catego- cream bar, or even the sound of a vacuum cleaner or coffee
ries reduces the amount of time used to search on a virtual machine, see Minsky and Fahey 2017). For example, Zampini
shelf, whether this sound is made by the packaging (e.g., the and Spence (2004) demonstrated that simply by manipulating
popping sound of the cork when a bottle of Champagne is the sounds made while biting into crisps (potato chips), the
opened), or a product-related jingle (e.g., the slogan of a perceived crispness and freshness of crisps can be enhanced.
laundry brand). Thus, congruent sounds may be used to help Similarly, Spence and Zampini (2007) found that the level and
the consumer find the products that they are looking for on a frequency of sounds can also affect a consumer's perception of
cluttered website more rapidly, even if this product does not the forcefulness (and hence efficacy) of aerosol sprays.
itself have any particularly salient sound associated with it. Meanwhile, Ho et al. (2013) were able to improve the virtual
However, since certain cross-modal associations can be experience associated with trying on new clothing (a product
contextually and culturally determined (e.g., flavor expecta- with more salient material properties) based on semantic
tions of colored beverage, Spence 2011; Wan et al. 2014), it is congruency, by adding synchronized naturalistic auditory
important for marketers to adapt the visual and auditory feedback. Immersed in the virtual clothing environment,
features not only to the products but also to the targeted participants imagined that they were out shopping for a winter
customers. jacket and had to try on two options in two conditions (with the
sound made by the clothes when the wearer moves were
synchronized vs. silence). The authors found that in the
presence of sound during the virtual trial, the users were
willing to pay more for the jacket than when they tried it on in
silence. Thus, by playing on the semantic congruency between
sounds and material properties in virtual environments,
marketers might help those consumers with a high NFT to be
more confident in their choices.
Care should, however, be taken not to bore the customer or
to assault their ears with too much auditory stimulation (what is
often referred to as “noise”; see Malhotra 1984; Spence 2014).
Consumers may prefer to shop on the Internet for the peace of
mind it provides, and the possibility of turning-off the sound
(interestingly, some bricks-and-mortar stores have now started
offering silent chill-out spaces: e.g., Selfridges, the London
department store offered this back in 2013; see Mardin 2013).
Nevertheless, while consumers may sometimes find back-
ground sound to be distracting, it is important to remember that
product sounds can provide an essential source of information
in terms of product evaluation (see Spence and Zampini 2006,
for a review). Moreover, marketers should be able to benefit
Fig. 4. MetaCookie+, Narumi et al. (2011). from new multisensory-enabling technologies to stimulate
O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61 53
several senses at once, which should further facilitate sensory (and that they are available for refill), which seems unrealistic
integration (Spence 2011; Stein and Stanford 2008). Some of at this stage.
these technologies are presented in the following section. One can also dream that it would be possible to share the
taste of products on the Internet (see Velasco et al. 2018, for a
Multisensory Online Experience review on multisensory technologies for online and mixed
reality food experiences). However, only a few SETs have
New multisensory devices are emerging, offering the succeeded in simulating the sense of taste (and mostly, only
opportunity to stimulate more of the customer's senses over within the confines of the technology labs; Straw-like User
the Internet. While these technologies are not yet fully Interface, Hashimoto, Inami, and Kajimoto 2008; food
commercialized, they let us dream of an online environment simulator, Iwata et al. 2004; Spence et al. 2017; Velasco et al.
more connected to the senses. For example, Ranasinghe et al. 2016a). Recently, Ranasinghe et al. (2017) presented a new
(2018) recently developed “Season Traveler,” a customized method by which to potentially share drinking experiences
wearable Head Mounted Display (HMD) system that features digitally over the Internet. First, they presented a device that is
smells, thermal, and wind stimuli to simulate real-world able to capture the color and pH value of a lemonade (among
environmental conditions when users explore (virtually) other liquids) and explained that these data might be digitally
different landscapes. Similarly, an AR device called transmitted to a special tumbler filled with plain water in
MetaCookie + allow the user to change the perceived flavor of another location. On receiving the information from the device,
food (e.g., a plain cookie) by virtually manipulating its the tumbler changes the color of the liquid in the glass using
appearance and diffusing additional smell (e.g., chocolate, LEDs and produced electrical stimulations on the user's tongue
strawberry; see Fig. 4, Narumi et al. 2011; see also Okajima and (note that the user has to stick their tongue out and touch it on
Spence 2011). the glass) with the hope of manipulating the experienced
New adaptations of sensory marketing strategies in online sourness of the ensuing taste sensation (see Fig. 5).
environments may be possible based on these multisensory On reading about the aforementioned techniques, marketers
interfaces. One day odors might be diffused while people are might imagine a future in which people could upload the flavor
online in order to stimulate perceptual re-enactment and (including taste and smell) before making their product choice
facilitate memorization and recall of information in the online in a supermarket or pizza home-delivery website. However,
environment (Braun et al. 2016; Krishna, Lwin, and Morrin before jumping straight into such futuristic scenarios, it is worth
2010; Morrin and Ratneshwar 2003). For instance, one could noting that there are many biological (e.g., individual
imagine the Doubletree chain of hotels diffusing the same scent differences in gustatory perception on the basis of thermal
of the cookies offered to customers at the reception desk, via stimulation of the tongue) and technical challenges (e.g., sweet
smell devices, during online booking, say, to set up an taste sensations are more difficult to elicit than sour or salty
anticipation of what's to come (though see Wan 2015). sensations) that need to be addressed before such systems
However, this requires that the devices have enough odors in become viable (see Spence et al. 2017). Such technologies are,
stock to be able to diffuse the one corresponding to the brand, then, not necessarily all that interesting to marketers in their
and that consumers also think of reloading the odor diffusers current form. That said, they do perhaps suggest new ways of
54 O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61
interacting with consumers online in the (near) future. Next, we How to Decide Whether Information Should Be Communicated
discuss opportunities for research. Through Visual, Haptic, Auditory, or Multisensory-Enabling
Technologies?
Need for Research
Depending on their objectives, marketers should consider
what kind of experience they wish to provide to consumers and
There have been many efforts in HCI directed at integrating
make a choice between different SETs. For instance, we
different sensory modalities online. However, further research
highlighted that visual-enabling technologies might serve to
is still needed in order to create more enjoyable and informative
make the online experience more immersive and enjoyable
multisensory experiences for the consumer by means of SETs.
(Animesh et al. 2011; Li et al. 2002), and that haptic interfaces
Marketing and HCI researchers should therefore think about
are useful when it comes to affecting the generosity of
what kind of experiences they wish to offer to consumers and
individuals (Spapé et al. 2015). Both technologies also appear
both capitalize on those new tools and develop others that
helpful in terms of facilitating product evaluation and purchase
facilitate the delivery of multisensory experiences. We try to
behavior, and might potentially one day be combined (Cano et
provide some answers to a series of outstanding relevant
al. 2017; Choi and Taylor 2014; Jin 2011). Other applications
questions, by suggesting some lines of research.
may undoubtedly be developed in order to improve the
interaction between online retailers and their customers (e.g.,
How Do Offline and Online Environments Differ in Terms of transaction uncertainty, Pavlou, Liang, and Xue 2007; engage-
Multisensory Information Processing? ment with the retailer, Shah and Murtaza 2005), or other
interlocutors (e.g., trust, satisfaction, and commitment between
Although the new SETs bring online and in-store environ- buyer and seller, Comer, Mehta, and Holmes 1998; see
ments closer together, the consumer's experiences are still not Varadarajan et al. 2010).
comparable. For instance, when consumers browse a retailer's Marketers should think carefully about their online sensory
website, the interaction is mainly through the screen of the needs and work jointly with HCI researchers on new interfaces
computer or the mobile phone, while in-store the consumer is that are more suited to consumers. For instance, Obrist, Seah,
completely immersed. Even if VR makes the online experience and Subramanian 2013 used ultrasound (mid-air haptics) in
more immersive (Animesh et al. 2011; Li, Daugherty, and order to create non-contact tactile sensations (associated by
Biocca 2002), people are not totally separated from their offline users with wind/breeze, textiles). At this stage, the interface has
environment (at home, at the office, at a terrace of a cafe). not been integrated into consumer experiences. Who knows,
Additionally, although these technologies provide sensory future versions might be developed in order to provide different
interactions with products (Cano et al. 2017; Jin 2011), their textile qualities (e.g., roughness, softness, elasticity), and might
effects on the online experience might not necessarily be thus help consumers to evaluate the clothes during online
identical to those in-store. Therefore, comparative studies shopping.
between the online and offline environments are still needed
(see Javornik 2016a, for a review). Further research might also How can One Assess the Optimal Personalized Multisensory
investigate the effects in terms of information processing Balance?
(visual attention, visual search, memorization, preferences, e.g.,
Knoeferle et al. 2016; Shen et al. 2016; Sunaga et al. 2016; Further research is needed in order to determine the right
Velasco et al. 2015), aspects of communication (e.g., balance in terms of the involvement of each sense in consumers'
connectivity, Calder, Malthouse, and Schaedel 2009; hyper- experiences. Too much sensory stimulation and one is in
textuality, Su 2008; interactivity, Song and Zinkhan 2008; and danger of creating “sensory overload” (Malhotra 1984; Raju
mobility, Sultan, Rohm, and Gao 2009), and also informative- 1980; Richard and Chebat 2016). It is not necessarily desirable
ness and entertainment (Childers et al. 2001; Eroglu et al. 2001; to always engage each and every one of the user's sense in order
Hsieh et al. 2014; Kim and Forsythe 2008a, b; Novak, to make an effective multisensory virtual display (Gallace et al.
Hoffman, and Yung 2000; Rose et al. 2012). 2012). Ultimately, the level of stimulation of each sense should
It might also be interesting to evaluate the extent to which perhaps be adapted to the “sensotype” of the individual (e.g.,
SETs stimulate mental imagery by facilitating the perceptual re- liking Lush/A&F-like olfactory rich environments vs. feeling
enactments of previous experiences, or otherwise reduce their sensory overload) and determined by the context, which could
relevance for consumers in the online purchase process. If encourage consumer acceptance of SETs (Dunn 2007; Wober
consumers can (virtually) touch, feel, or taste the products by 1991). For example, vision might dominate when the geometric
means of SETs, mental imagery might not be necessary aspects (size, orientation) of the products are relevant for its
anymore. Moreover, it would be (virtually) possible to taste a evaluation, and haptic/smell/taste might be considered when it
product before (or without) smelling or touching it, which comes to a product's material properties (Gallace et al. 2012).
could potentially change the psychological distance (i.e., make Personalizing online information can be expensive for
the subjective experience that it is close regardless of the actual brands. Therefore, the latter should question what individual
physical distance), with products on the Internet (Elder et al. differences in multisensory perception might be interesting to
2017). consider, and when and how multisensory experiences should
O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61 55
Fig. 6. (a.) Tesco virtual supermarket in a subway station created by Cheil (b.) Factual augmented supermarket created by Keiichi MATSUDA.
be personalized in online environments. For example, it may Grewal 1992; Kotler, 1973; Turley and Milliman, 2000).
not be necessary to customize the interaction of consumers with Diffusing a pleasant odor, color, or music can contribute to the
a watch (i.e., a product with salient geometric properties) based positive evaluation of the store and influences the time and the
on their NFT. It should also be noted that consumers are money that consumers spend there (Chebat and Michon, 2003;
currently not used to manipulate the new SETs on the Internet. Sherman, Mathur, and Smith 1997; Spangenberg, Crowley, and
Therefore, further studies should also be conducted to Pamela 1996). In addition, the more store atmospherics are
understand any novelty effects, as well as how to facilitate the multisensory and congruent through the senses, the more
acceptance these new technologies. pleasant and interesting for consumers they are evaluated (see
Spence et al. 2014, for a review).
How to Better Connect the Online and the Offline Several interactive technologies, such as shopping assistant
Environments? systems and smart mirrors, are already modifying the
traditional store experience (Brasel and Gips 2014; Cano et al.
In a traditional retail context, the atmospherics created 2017; see Pantano and Naccarato 2010, for a review). For
notably by sensory environmental cues (e.g., color, lighting, instance, touch screens can facilitate the interaction with the
music, scent) have been shown to influence the behavior of product and create stronger perceived ownership, enabling
customers through their emotional reactions (Baker, Levy, and extended use both in-store and out to the store (Brasel and Gips,
56 O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61
2014; Cano et al., 2017). Technologies are becoming an experience not only in “the real world,” but also in a range of
increasingly important part of store atmosphere and an effective online environments. Including sensory information via
means of luring consumers in to the store. In this way, Poncin websites is all the more important given that it results in
and Mimoun (2014) highlighted that using magic mirrors with consumers being more confident in their choices and increases
augmented reality and interactive game terminals in a physical the likelihood that a liked product will be purchased. However,
store has a positive effect on the perception of store consumers do not necessarily need to touch or smell the
atmospherics. products in order to get the relevant sensory information. They
The SETs offer the customer the opportunity to browse can also imagine the expected sensory properties of the
online from the store, getting more information about products, products based on their previous product experiences, with
in addition to having a positive effect on the perception of a the support of basic digital interfaces (e.g., screen, mouse, and
store's atmosphere (Kent et al. 2015). However, SETs should headphones). Moreover, recent progress in HCI suggests that at
not simply be seen as tools with which to connect the online least new visual- and haptic-enabling technologies should be
environment with the physical store, but also as a means to available on the Internet soon. Hopefully, these technologies
create new environments in which physical and virtual objects/ will go beyond simply reinforcing the effects of sensory
products coexist and potentially interact (Milgram and Kishino marketing strategies on consumer's online behavior, but also
1994). For instance, Pokemon Go revealed a bright new future create new forms of interaction taking place not only in virtual
in which the borders between the real and imaginary world are or in the real places but in mixed reality environments too.
no longer so clearly delineated (Milgram and Kishino 1994). In We have presented a selective list of potential sensory
a mixed reality environment, people might be more easily technological developments for digital environments. We have
detached from reality than from a simple in-store experience also raised some of their limits (distractive, untrustworthy, and
(Javornik 2016a). sensory overload). We have highlighted some of the ways in
Marketers should think about building new spaces for which marketers can use these innovations to better transfer
interaction with consumers through mixed reality and propos- sensory information to consumers in the online environment.
ing new modes of experiential consumption (see Petit, Velasco, Here, it is worth remembering that many SETs currently only
and Spence 2018, for a review on digital multisensory exist as prototypes (and hence people are not necessarily
packaging). The virtual grocery store opened by Tesco in a accustomed to them), while others are still to be invented! For
South Korean subway station provides a good example of this reason, the objective was not to describe and delimit what
mixed reality. The glass walls of certain subway stations were is exactly digital sensory marketing, but rather to provide a
covered with images of supermarket shelfs (including products, greater understanding of its interests for the future of marketing
prices, and bare codes), and commuters were able to shop using research. Many challenges and questions await marketers and
their smartphones (see Fig. 6a). Another example comes from researchers in the integration of sensory marketing in the digital
Keiichi Matsuda, who proposes a glimpse of a world in which world. For us, these challenges mainly revolve around finding
all the elements of everyday life are enhanced through the eyes the right balance between the different sensory inputs that
of a woman (e.g., a supermarket, in which an avatar pet on a might be stimulated online and/or offline in mixed reality, and
shopping trolley offers discounts, see Fig. 6b). Through mixed potentially adapted to an individual's preferences, and location
reality, people might share the same physical space and have (e.g., at home, at the office, in a physical store). We believe that
different AR contents. They might also view the same AR layer it is important that marketers become aware of this new
while dispersed across different locations (Scholz and Smith evolution to anticipate and analyze how new technologies
2016). Therefore, it is important to understand whether these will impact market attitudes and behaviors through the
situations are similar in terms of embodiment (Wilson 2002). “sensorialization” of digital environments.
To finish, it remains to be determined what level of hyper-
connectivity and realism would be acceptable and beneficial to
consumers. According to Belk (1988), persons, places, and References
things to which one feels attached are part of the extended self. If
Ablart, Damien, Carlos Velasco, and Marianna Obrist (2017, June), “Integrat-
the SETs offer new opportunities for people to extend their self ing Mid-Air Haptics Into Movie Experiences,” paper presented at
through the possession of digital objects, above all, they International Conference on Interactive Experiences for Television and
highlight a disappearance of the boundaries between consumers, Online Video, Netherlands: Hilversum.
products, and brands (Belk 2013). The objects of the mixed Algharabat, Raed, Ali Abdallah Alalwan, Nripendra P. Rana, and Yogesh K.
reality will become more embodied, invisible, to constitute a Dwivedi (2017), “Three Dimensional Product Presentation Quality
Antecedents and Their Consequences for Online Retailers: The Moderating
natural part of the self (Belk 2014). Therefore, further studies Role of Virtual Product Experience,” Journal of Retailing and Consumer
should explore how digital products can match consumers' Services, 36, 203–17.
expectations of themselves (Scholz and Duffy 2018). Animesh, Animesh, Alain Pinsonneault, Sung-Byung Yang, and Wonseok Oh
(2011), “An Odyssey into Virtual Worlds: Exploring the Impacts of
Conclusions Technological and Spatial Environments on Intention to Purchase Virtual
Products,” MIS Quarterly, 35, 3, 789–810.
Armel, K. Carrie, Aurelie Beaumel, and Antonio Rangel (2008), “Biasing
In this review, we have highlighted the key role that Simple Choices by Manipulating Relative Visual Attention,” Judgment and
multisensory information has in mediating consumer Decision Making, 3, 5, 396–403.
O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61 57
Baker, Julie, Michael Levy, and Dhruv Grewal (1992), “An Experimental Choi, Yung Kyun and Charles R. Taylor (2014), “How do 3-Dimensional
Approach to Making Retail Store Environmental Decisions,” Journal of Images Promote Products on the Internet?” Journal of Business Research,
Retailing, 68, 4, 445–60. 67, 10, 2164–70.
Barsalou, Lawrence W. (2008), “Grounded Cognition,” Annual Review of Christian, Brittany M., Lynden K. Miles, Sophie T. Kenyeri, Jennifer
Psychology, 59, 617–45. Mattschey, and C. Neil Macrae (2016), “Taming Temptation: Visual
Basso, Frédéric, Olivia Petit, Sophie Le Bellu, Saadi Lahlou, A. Aïda Perspective Impacts Consumption and Willingness to Pay for Unhealthy
Cancel, and Jean-Luc Anton (2018), “Taste at First (Person) Sight: Foods,” Journal of Experimental Psychology: Applied, 22, 1, 85–94.
Visual Perspective Modulates BMI-Dependent Brain Activity Associ- Chung, Sorim, Thomas Kramer, and Elaine M. Wong. (2018), "Do Touch
ated With Viewing Unhealthy but not Healthy Foods,” Appetite, 128, Interface Users Feel More Engaged? The Impact of Input Device Type on
242–54. Online Shoppers’ Engagement, Affect, and Purchase Decisions," Psychol-
Beck, Marie and D. Crié (2018), “I Virtually Try It… I Want It! Virtual Fitting ogy & Marketing.
Room: A Tool to Increase On-Line and Off-Line Exploratory Behavior, Cian, Luca, Aradhna Krishna, and Ryan S. Elder (2014), “This Logo Moves
Patronage and Purchase Intentions,” Journal of Retailing and Consumer Me: Dynamic Imagery From Static Images,” Journal of Marketing
Services, 40, 279–86. Research, 51, 2, 184–97.
Belk, Russell W. (1988), “Possessions and the Extended Self,” Journal of Comer, James M., Raj Mehta, and Terence L. Holmes (1998), “Information
Consumer Research, 15, 2, 139–68. Technology: Retail Users Versus Nonusers,” Journal of Interactive
——— (2013), “Extended Self in a Digital World,” Journal of Consumer Marketing, 12, 2, 49–62.
Research, 40, 3, 477–500. Crusco, April H. and Christopher G. Wetzel (1984), “The Midas Touch the
——— (2014), “Digital Consumption and the Extended Self,” Journal of Effects of Interpersonal Touch on Restaurant Tipping,” Personality and
Marketing Management, 30, 11–12, 1101–18. Social Psychology Bulletin, 10, 4, 512–7.
Bölte, Jens, Thomas M. Hösker, Gerrit Hirschfeld, and Meinald Thielsch Dawar, Niraj and Philip Parker (1994), “Marketing Universals: Consumers' Use
(2017), “Electrophysiological Correlates of Aesthetic Processing of of Brand Name, Price, Physical Appearance, and Retailer Reputation as
Webpages: A Comparison of Experts and Laypersons,” PeerJ, 5e3440. Signals of Product Quality,” Journal of Marketing, 58, 2, 81–95.
Brasel, S. Adam and James Gips (2014), “Tablets, Touchscreens, and De Volder, Anne G., Hinako Toyama, Yuichi Kimura, Motohiro Kiyosawa,
Touchpads: How Varying Touch Interfaces Trigger Psychological Owner- Hideki Nakano, Annick Vanlierde, Marie-Chantal Wanet-Defalque,
ship and Endowment,” Journal of Consumer Psychology, 24, 2, 226–33. Masahiro Mishina, Keiichi Oda, Kiichi Ishiwata, and Michio Senda
——— and ——— (2015), “Interface Psychology: Touchscreens Change (2001), “Auditory Triggered Mental Imagery of Shape Involves Visual
Attribute Importance, Decision Criteria, and Behavior in Online Choice,” Association Areas in Early Blind Humans,” NeuroImage, 14, 1, 129–39.
Cyberpsychology, Behavior, and Social Networking, 18, 9, 534–8. Dinh, Huong Q., Neff Walker, Chang Song, Akira Kobayashi, and Larry F.
Braun, Marius H., Gilang Andi Pradana, George Buchanan, Adrian D. Cheok, Hodges (1999, March), “Evaluating the Importance of Multi-Sensory Input
Carlos Velasco, Charles Spence, Andoni Luis Aduriz, Jade Gross, and on Memory and the Sense of Presence in Virtual Environments,” Virtual
Dani Lasa (2016), “Emotional Priming of Digital Images Through Mobile Reality, IEEE, 222–8.
Tele-Smell and Virtual Food,” International Journal of Food Design, 1, Dunn, Winnie (2007), Living Sensationally: Understanding Your Senses.
29–45. London, UK: Jessica Kingsley Publishers.
Brengman, Malaika, Kim Willems, and Helena Van Kerrebroeck (2018), “Can't Eelen, Jiska, Siegfried Dewitte, and Luk Warlop (2013), “Situated Embodied
Touch This: The Impact of Augmented Reality Versus Touch and Non- Cognition: Monitoring Orientation Cues Affects Product Evaluation and
Touch Interfaces on Perceived Ownership,” Virtual Reality, 1–12. Choice,” Journal of Consumer Psychology, 23, 4, 424–33.
Bridges, Eileen and Renée Florsheim (2008), “Hedonic and Utilitarian Elder, Ryan S. and Aradhna Krishna (2012), “The “Visual Depiction Effect” in
Shopping Goals: The Online Experience,” Journal of Business Research, Advertising: Facilitating Embodied Mental Simulation Through Product
61, 4, 309–14. Orientation,” Journal of Consumer Research, 38, 6, 988–1003.
Calder, Bobby J., Edward C. Malthouse, and Ute Schaedel (2009), “An ———, Anne E. Schlosser, Morgan Poor, and L. Lidan Xu (2017), “So Close I
Experimental Study of the Relationship Between Online Engagement and Can Almost Sense It: The Interplay Between Sensory Imagery and
Advertising Effectiveness,” Journal of Interactive Marketing, 23, 4, Psychological Distance,” Journal of Consumer Research, 44, 4, 877–94.
321–31. Eroglu, Sevgin A., Karen A. Machleit, and Lenita M. Davis (2001),
Calvert, Gemma A., Edward T. Bullmore, Michael J. Brammer, Ruth Campbell, “Atmospheric Qualities of Online Retailing: A Conceptual Model and
Steven C.R. Williams, Philip K. McGuire, Peter W.R. Woodruff, Susan Implications,” Journal of Business Research, 54, 2, 177–84.
Iversen, and Anthony S. David (1997), “Activation of Auditory Cortex Favre, Jean Paul and André November (1979), Color and Communication.
During Silent Lipreading,” Science, 276, 5312, 593–6. Zurich: ABC-Verlag.
Cano, Marta Blazquez, Patsy Perry, Rachel Ashman, and Kathryn Waite Feng, Mi, Arindam Dey, and Robert W. Lindeman (2016, March), “An Initial
(2017), “The Influence of Image Interactivity Upon User Engagement Exploration of a Multi-Sensory Design Space: Tactile Support for Walking
When Using Mobile Touch Screens,” Computers in Human Behavior, 77, in Immersive Virtual Environments,” 3D User Interfaces (3DUI), IEEE,
406–12. 95–104.
Chebat, Jean-Charles and Richard Michon (2003), “Impact of Ambient Odors on Fransson, Peter and Guillaume Marrelec (2008), “The Precuneus/Posterior
Mall Shoppers' Emotions, Cognition, and Spending: A Test of Competitive Cingulate Cortex Plays a Pivotal Role in the Default Mode Network:
Causal Theories,” Journal of Business Research, 56, 7, 529–39. Evidence from a Partial Correlation Network Analysis,” NeuroImage, 42, 3,
Chen, Jing, Esther K. Papies, and Lawrence W. Barsalou (2016), “A Core 1178–84.
Eating Network and Its Modulations Underlie Diverse Eating Phenomena,” Gabisch, Jason A. (2011), “Virtual World Brand Experience and its Impact on
Brain and Cognition, 110, 20–42. Real World Purchasing Behaviour,” Journal of Brand Management, 19, 1,
Chen, Yi-Chuan and Charles Spence (2018), “Audiovisual Semantic Interac- 18–32.
tions Between Linguistic and Non-Linguistic Stimuli: The Time-Courses Gallace, Alberto, Mary K. Ngo, John Sulaitis, and Charles Spence (2012),
and Categorical Specificity,” Journal of Experimental Psychology: Human “Multisensory Presence in Virtual Reality: Possibilities & Limitations,”
Perception and Performance. Multiple Sensorial Media Advances and Applications: New Developments
Childers, Terry L., Christopher L. Carr, Joann Peck, and Stephen Carson in MulSeMedia, Hershey, PA: IGI Global, 1–38.
(2001), “Hedonic and Utilitarian Motivations for Online Retail Shopping ——— and Charles Spence (2014), In Touch with the Future: The Sense of
Behaviour,” Journal of Retailing, 77, 4, 511–35. Touch from Cognitive Neuroscience to Virtual Reality. Oxford, UK: Oxford
Cho, Hyejeung and Norbert Schwarz (2012), “I Like Your Product When I Like University Press.
My Photo: Misattribution Using Interactive Virtual Mirrors,” Journal of González, Julio, Alfonso Barros-Loscertales, Friedemann Pulvermüller,
Interactive Marketing, 26, 4, 235–43. Vanessa Meseguer, A. Ana Sanjuán, Vicente Belloch, and César Ávila
58 O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61
(2006), “Reading Cinnamon Activates Olfactory Brain Regions,” Design: Experiential In-Store Digital Environments,” in Successful
NeuroImage, 32, 2, 906–12. Technological Integration for Competitive Advantage in Retail Settings.
Guerrero, Laura K. and Peter A. Andersen (1994), “Patterns of Matching and E. Pantano, editor. Hershey, PA: Business Science Reference, 1–22.
Initiation: Touch Behavior and Touch Avoidance Across Romantic Kim, Jiyeon and Sandra Forsythe (2008a), “Sensory Enabling Technology
Relationship Stages,” Journal of Nonverbal Behavior, 18, 2, 137–53. Acceptance Model (SE-TAM): A Multiple-Group Structural Model
Gvili, Yaniv, Aner Tal, Moty Amar, and Brian Wansink (2017), “Moving up in Comparison,” Psychology & Marketing, 25, 9, 901–22.
Taste: Enhanced Projected Taste and Freshness of Moving Food Products,” ——— and ——— (2008b), “Adoption of Virtual Try-On Technology for
Psychology & Marketing, 34, 7, 671–83. Online Apparel Shopping,” Journal of Interactive Marketing, 22, 2, 45–59.
Haans, Antal and Wijnand A. IJsselsteijn (2009), “The Virtual Midas Touch: Kim, Sang-Youn, Ki-baek Kim, Jaehwan Kim, and Ki-uk Kyung (2013), “A
Helping Behavior After a Mediated Social Touch,” IEEE Transactions on Film-Type Vibrotactile Actuator for Hand-Held Devices,” in Haptic and
Haptics, 2, 3, 136–40. Audio Interaction Design, 7989. I. Oakley, S. Brewster, editors. LNCS,
———, Renske de Bruijn, and Wijnand A. IJsselsteijn (2014), “A Virtual Berlin: Springer, 109–16.
Midas touch? Touch, Compliance, and Confederate Bias in Mediated Kitagawa, Norimichi and Yuka Igarashi (2005), “Tickle Sensation Induced
Communication,” Journal of Nonverbal Behavior, 38, 3, 301–11. by Hearing a Sound (Summary of Awarded Presentation at the 23rd
Hagtvedt, Henrik and S.A. Brasel (2016), “Cross-Modal Communication: Annual Meeting),” The Japanese Journal of Psychonomic Science, 24, 1,
Sound Frequency Influences Consumer Responses to Color Lightness,” 121–2.
Journal of Marketing Research, 53, 4, 551–62. Klein, Lisa R. (2003), “Creating Virtual Product Experiences: The Role of
Hashimoto, Yuki, Masahiko Inami, and Hiroyuki Kajimoto (2008, June), Telepresence,” Journal of Interactive Marketing, 17, 1, 41–55.
“Straw-Like User Interface (Ii): A New Method of Presenting Auditory Knoeferle, Klemens M., Pia Knoeferle, Carlos Velasco, and Charles Spence
Sensations for a More Natural Experience,” .paper presented at Interna- (2016), “Multisensory Brand Search: How the Meaning of Sounds Guides
tional Conference on Human Haptic Sensing and Touch Enabled Computer Consumers' Visual Attention,” Journal of Experimental Psychology:
Applications, Berlin: Springer, 484–93. Applied, 22, 2, 196–210.
Hertenstein, Matthew J., Dacher Keltner, Betsy App, Brittany Bulleit, and Knutson, Brian, Rick Scott, G. Elliott Wimmer, Drazen Prelec, and George
Ariane R. Jaskolka (2006), “Touch Communicates Distinct Emotions,” Loewenstein (2007), “Neural Predictors of Purchases,” Neuron, 53, 1,
Emotion, 6, 3, 528–33. 147–56.
Ho, Cristy, Russ Jones, Scott King, Lynne Murray, and Charles Spence (2013), Kotler, Philip (1973), “Atmospherics as a Marketing Tool,” Journal of
“Multisensory Augmented Reality in the Context of a Retail Clothing Retailing, 49, 4, 48–64.
Application,” in (((ABA))) Audio Branding Academy Yearbook 2012/2013. Kreijns, Karel, Paul A. Kirschner, Wim Jochems, and Hansvan Buurenc (2007),
K. Bronner, R. Hirt, C. Ringe, editors. Baden-Baden: Nomos, 167–75. “Measuring Perceived Sociability of Computer-Supported Collaborative
Hornik, Jacob (1992), “Tactile Stimulation And Consumer Response,” Journal Learning Environments,” Computers & Education, 49, 2, 176–92.
of Consumer Research, 19, 3, 449–58. Krishna, Aradhna, May O. Lwin, and Maureen Morrin (2010), “Product Scent
Hsieh, Jung-Kuei, Yi-Ching Hsieh, Hung-Chang Chiu, and Ya-Ru Yang and Memory,” Journal of Consumer Research, 37, 1, 57–67.
(2014), “Customer Response to Web Site Atmospherics: Task-Relevant ——— (2012), “An Integrative Review of Sensory Marketing: Engaging the
Cues, Situational Involvement and PAD,” Journal of Interactive Marketing, Senses to Affect Perception, Judgment and Behaviour,” Journal of
28, 3, 225–36. Consumer Psychology, 22, 3, 332–51.
Huang, Echo (2012), “Online Experiences and Virtual Goods Purchase ——— and Norbert Schwarz (2014), “Sensory Marketing, Embodiment, and
Intention,” Internet Research, 22, 3, 252–74. Grounded Cognition: A Review and Introduction,” Journal of Consumer
Huang, Tseng-Lung and Shu-Ling Liao (2017), “Creating e-Shopping Psychology, 24, 2, 159–68.
Multisensory Flow Experience Through Augmented-Reality Interactive ———, Luca Cian, and Tatiana Sokolova (2016), “The Power of Sensory
Technology,” Internet Research, 27, 2, 449–75. Marketing in Advertising,” Current Opinion in Psychology, 10, 142–7.
Huisman, Gijs (2017), “Social Touch Technology: A Survey of Haptic ———, ———, and Nilüfer Z. Aydınoğlu (2017), “Sensory Aspects of
Technology for Social Touch,” IEEE Transactions on Haptics, 99 Package Design,” Journal of Retailing, 93, 1, 43–54.
1939–1412. Lacey, Simon, Peter Flueckiger, Randall Stilla, Lava Michael, and K. Sathiana
Huxley, Aldous (1932), Brave New World. London, UK: Chatto & Windus. (2010), “Object Familiarity Modulates the Relationship Between Visual
Iwata, Hiroo, Hiroaki Yano, Takahiro Uemura, and Tetsuro Moriya (2004, Object Imagery and Haptic Shape Perception,” NeuroImage, 49, 3,
March), “Food Simulator: A Haptic Interface for Biting,” Virtual Reality, 1977–90.
IEEE, 51–7. Lederman, Susan J. (1974), “Tactile Roughness of Grooved Surfaces: The
Jai, Tun-Min, Michael W. O'Boyle, and Dan Fang (2014), “Neural Correlates Touching Process and Effects of Macro- and Microsurface Structure,”
of Sensory-Enabling Presentation: An fMRI Study of Image Zooming and Perception & Psychophysics, 16, 385–95.
Rotation Video Effects on Online Apparel Shopping,” Journal of Consumer Lee, Kun Chang and Namho Chung (2008), “Empirical Analysis of Consumer
Behaviour, 13, 5, 342–50. Reaction to the Virtual Reality Shopping Mall,” Computers in Human
Javornik, Ana (2016a), “Augmented Reality: Research Agenda for Studying the Behavior, 24, 1, 88–104.
Impact of its Media Characteristics on Consumer Behaviour,” Journal of Leithinger, Daniel, Sean Follmer, Alex Olwal, and Hiroshi Ishii (2014,
Retailing and Consumer Services, 30, 252–61. October), “Physical Telepresence: Shape Capture and Display for
——— (2016b), “‘It's an Illusion, But it Looks Real!’ Consumer Affective, Embodied, Computer-Mediated Remote Collaboration,” Proceedings of
Cognitive and Behavioural Responses to Augmented Reality Applications,” the 27th Annual ACM Symposium on User Interface Software and
Journal of Marketing Management, 32, 9–10, 987–1011. Technology, ACM, 461–70.
Jiang, Zhenhui and Izak Benbasat (2004), “Virtual Product Experience: Effects Li, Hairong, Terry Daugherty, and Frank Biocca (2001), “Characteristics of
of Visual and Functional Control of Products on Perceived Diagnosticity Virtual Experience in Electronic Commerce: A Protocol Analysis,” Journal
and Flow in Electronic Shopping,” Journal of Management Information of Interactive Marketing, 15, 3, 13–30.
Systems, 21, 3, 111–47. ———, ———, and ——— (2002), “Impact of 3-D Advertising on Product
Jin, Seung-A Annie (2009), “The Roles of Modality Richness and Involvement Knowledge, Brand Attitude, and Purchase Intention: The Mediating Role of
in Shopping Behavior in 3D Virtual Stores,” Journal of Interactive Presence,” Journal of Advertising, 31, 3, 43–57.
Marketing, 23, 3, 234–46. Liu, Rebecca, Mackenzie Hannum, and Christopher T. Simons (2018), Using
——— (2011), “The Impact of 3D Virtual Haptics in Marketing,” Psychology Immersive Technologies to Explore the Effects of Congruent and
& Marketing, 28, 3, 240–55. Incongruent Contextual Cues on Context Recall, Product Evaluation
Kent, Anthony, Charles Dennis, Marta Blazquez Cano, Schwarz Eva, J. Joško Time, and Preference and Liking During Consumer Hedonic Testing.
Brakus, and Eleftherios Alamanos (2015), “Branding, Marketing, and Food Research International.
O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61 59
Malhotra, Naresh K. (1984), “Reflections on the Information Overload Papies, Esther K. and Lawrence Barsalou (2015), “Grounding Desire and
Paradigm in Consumer Decision Making,” Journal of Consumer Research, Motivated Behavior: A Theoretical Framework and Review of Empirical
10, 4, 436–40. Evidence,” in The Psychology of Desire. W. Hofmann, L. Nordgren, editors.
Mardin, Sandra (2013), “How Selfridges Made Noise About Being Quiet,” New York, NY: Guilford Press.
Canvas, 8Downloaded from http://www.canvas8.com/public/2013/01/18/ Park, Jihye, Leslie Stoel, and Sharron J. Lennon (2008), “Cognitive, Affective
no-noise-selfridges.html. and Conative Responses to Visual Simulation: The Effects of Rotation in
Marks, Lawrence E. (1987), “On Cross-Modal Similarity: Auditory–Visual Online Product Presentation,” Journal of Consumer Behaviour, 7, 1, 72–87.
Interactions in Speeded Discrimination,” Journal of Experimental Psychol- Park, Young-Woo and Tek-Jin Nam (2013), “Poke: A New Way of Sharing
ogy: Human Perception and Performance, 13, 3, 384. Emotional Touches During Phone Conversations,” CHI’13 Extended
Mattila, Anna S. and Jochen Wirtz (2001), “Congruency of Scent and Music as Abstracts on Human Factors in Computing Systems, ACM, 2859–60.
a Driver of In-Store Evaluations and Behavior,” Journal of Retailing, 77, 2, Pavlou, Paul A., Huigang Liang, and Yajiong Xue (2007), “Understanding and
273–89. Mitigating Uncertainty in Online Exchange Relationships: A Principal-
McCabe, Deborah Brown and Stephen M. Nowlis (2003), “The Effect of Agent Perspective,” MIS Quarterly, 31, 1, 105–36.
Examining Actual Products or Product Descriptions on Consumer Peck, Joann and Terry L. Childers (2003a), “To Have and to Hold: The
Preference,” Journal of Consumer Psychology, 13, 4, 431–9. Influence of Haptic Information on Product Judgments,” Journal of
Merle, Aurélie, Sylvain Senecal, and Anik St-Onge (2012), “Whether and How Marketing, 67, 2, 35–48.
Virtual Try-On Influences Consumer Responses to an Apparel Web Site,” ——— and ——— (2003b), “Individual Differences in Haptic Information
International Journal of Electronic Commerce, 16, 3, 41–64. Processing: The “Need for Touch” Scale,” Journal of Consumer Research,
Milgram, Paul and Fumio Kishino (1994), “A Taxonomy of Mixed Reality 30, 3, 430–42.
Visual Displays,” IEICE Transactions on Information and Systems, 77, 12, ——— and Suzanne B. Shu (2009), “The Effect of Mere Touch on Perceived
1321–9. Ownership,” Journal of Consumer Research, 36, 3, 434–47.
Milosavljevic, Milica, Vidhya Navalpakkam, Christof Koch, and Antonio ———, Victor A. Barger, and Andrea Webb (2013), “In Search of a Surrogate
Rangel (2012), “Relative Visual Saliency Differences Induce Sizable Bias for Touch: The Effect of Haptic Imagery on Perceived Ownership,” Journal
in Consumer Choice,” Journal of Consumer Psychology, 22, 67–74. of Consumer Psychology, 23, 2, 189–96.
Minsky, Laurence and Colleen Fahey (2017), Audio Branding: Using Sound to Pessoa, Luiz and Peter De Weerd (2003), Filling-In: From Perceptual
Build Your Brand. London, UK: Kogan Page. Completion to Cortical Reorganization. Oxford University Press.
Morrin, Maureen and Srinivasan Ratneshwar (2003), “Does it Make Sense to Petit, Olivia, Adrian D. Cheok, Charles Spence, Carlos Velasco, and Kasun T.
Use Scents to Enhance Brand Memory?” Journal of Marketing Research, Karunanayaka (2015, November), “Sensory Marketing in Light of New
40, 1, 10–25. Technologies,” .proceedings from International Conference on Advances in
Mueller, Florian F., Frank Vetere, Martin R. Gibbs, Jesper Kjeldskov, Sonja Computer Entertainment Technology, ACM, 53.
Pedell, and Steve Howard (2005), “Hug Over a Distance,” CHI 05, ACM, ———, Frédéric Basso, Dwight Merunka, Charles Spence, Adrian D. Cheok,
1673–6. and Olivier Oullier (2016), “Pleasure and the Control of Food Intake: An
Nah, Fiona Fui-Hoon, Brenda Eschenbrenner, and Davis Dewester (2011), Embodied Cognition Approach to Consumer Self-Regulation,” Psychology
“Enhancing Brand Equity Through Flow and Telepresence: A Comparison & Marketing, 33, 8, 608–19.
of 2D and 3D Virtual Worlds,” MIS Quarterly, 35, 3, 731–47. ———, Charles Spence, Carlos Velasco, Andy T. Woods, and A.D. Cheok
Narumi, Takuji, Shinya Nishizaka, Takashi Kajinami, Tomohiro Tanikawa, and (2017), “Changing the Influence of Portion Size on Consumer Behavior Via
Michitaka Hirose (2011, May), “Augmented Reality Flavors: Gustatory Imagined Consumption,” Journal of Business Research, 75, 240–8.
Display Based on Edible Marker and Cross-Modal Interaction,” .paper ———, Carlos Velasco, and Charles Spence (2018), “Multisensory Consumer-
presented at SIGCHI Conference on Human Factors in Computing Systems, Packaging Interaction (CPI): The Role of New Technologies,” in
ACM, 93–102. Multisensory Packaging: Designing New Product Experiences. Carlos
Niedenthal, Paula M., Lawrence W. Barsalou, Piotr Winkielman, Silvia Krauth- Velasco, Spence Charles, editors. New York, NY: Palgrave MacMillan.
Gruber, and François Ric (2005), “Embodiment in Attitudes, Social Poncin, Ingrid and Mohamed Slim Ben Mimoun (2014), “The Impact of “e-
Perception, and Emotion,” Personality and Social Psychology Review, 9, Atmospherics” on Physical Stores,” Journal of Retailing and Consumer
3, 184–211. Services, 21, 5, 851–9.
Novak, Thomas P., Donna Hoffman, and Yiu-Fai Yung (2000), “Measuring the Poushneh, Atieh and Arturo Z. Vasquez-Parraga (2017), “Discernible Impact of
Customer Experience in Online Environments: A Structural Modeling Augmented Reality on Retail Customer's Experience, Satisfaction and
Approach,” Marketing Science, 19, 1, 22–42. Willingness to Buy,” Journal of Retailing and Consumer Services, 34,
Obrist, Marianna, Sue Ann Seah, and Sriram Subramanian (2013, April), 229–34.
“Talking About Tactile Experiences,” .paper presented at Raju, Puthankurissi S. (1980), “Optimum Stimulation Level: Its Relationship to
SIGCHI Conference on Human Factors in Computing Systems, ACM, Personality, Demographics, and Exploratory Behaviour,” Journal of
1659–68. Consumer Research, 7, 3, 272–82.
———, Carlos Velasco, Chi Vi, Nimesha Ranasinghe, Ali Israr, Adrian D. Ranasinghe, Nimesha, Pravar Jain, Shienny Karwita, and Ellen Yi-Luen Do
Cheok, Charles Spence, and Ponnampalam Gopalakrishnakone (2016, (2017, March), “Virtual Lemonade: Let's Teleport Your Lemonade!”, paper
May), “Touch, Taste, & Smell User Interfaces: The Future of Multisensory presented at International Conference on Tangible, Embedded, and
HCI,” .paper presented at CHI Conference Extended Abstracts on Human Embodied Interaction (pp. 183–190), ACM.
Factors in Computing Systems, ACM, 3285–92. Ranasinghe, Nimesha, Pravar Jain, Nguyen Thi Ngoc Tram, Koon C.R. Koh,
Okajima, Katsunori, Adrian D. Cheok, O. Olivia Petit, and Charles Michel David Tolley, Shienny Karwita, and Ellen Yi-Luen Do (2018, April),
(2016), “Eating with Our Eyes: From Visual Hunger to Digital Satiation,” “Season Traveller: Multisensory Narration for Enhancing the Virtual
Brain and Cognition, 110, 53–63. Reality Experience,” .paper presented at CHI Conference on Human
——— and Charles Spence (2011), “Effects of Visual Food Texture on Taste Factors in Computing Systems, ACM, 577.
Perception,” i-Perception, 2, 8http://i-perception.perceptionweb.com/ Rantala, Jussi, Katri Salminen, Roope Raisamo, and Veikko Surakka (2013),
journal/I/article/ic966. “Touch Gestures in Communicating Emotional Intention via Vibrotactile
Overby, Jeffrey W. and Eun-Ju Lee (2006), “The Effects of Utilitarian and Stimulation,” International Journal of Human-Computer Studies, 71, 6,
Hedonic Online Shopping Value on Consumer Preference and Intentions,” 679–90.
Journal of Business Research, 59, 10–11, 1160–6. Richard, Marie-Odile and Jean-Charles Chebat (2016), “Modeling Online
Pantano, Eleonora and Giuseppe Naccarato (2010), “Entertainment in Retailing: Consumer Behavior: Preeminence of Emotions and Moderating Influences
The Influences of Advanced Technologies,” Journal of Retailing and of Need for Cognition and Optimal Stimulation Level,” Journal of Business
Consumer Services, 17, 3, 200–4. Research, 69, 2, 541–53.
60 O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61
Rose, Susan, Moira Clark, Phillip Samouel, and Neil Hair (2012), “Online Statista (2016), “North America Daily Time with Internet 2016,” Retrieved
Customer Experience in e-Retailing: An Empirical Model of Antecedents from https://www.statista.com/statistics/645644/north-america-daily-time-
and Outcomes,” Journal of Retailing, 88, 2, 308–22. per-capita-internet/.
Sallnäs, Eva-Lotta, Kirsten Rassmus-Gröhn, and Calle Sjöström (2000), “Supporting Stein, Barry E. and Terrence R. Stanford (2008), “Multisensory Integration:
Presence in Collaborative Environments by Haptic Force Feedback,” ACM Current Issues from the Perspective of the Single Neuron,” Nature Reviews
Transactions on Computer-Human Interaction (TOCHI), 7, 4, 461–76. Neuroscience, 9, 4, 255.
Scholz, Joachim and Andrew N. Smith (2016), “Augmented Reality: Designing Su, Bo-Chiuan (2008), “Characteristics of Consumer Search On-Line: How
Immersive Experiences that Maximize Consumer Engagement,” Business Much Do We Search?” International Journal of Electronic Commerce, 13,
Horizons, 59, 2, 149–61. 1, 109–29.
——— and Katherine Duffy (2018), “We ARe at Home: How Augmented Sultan, Fareena, Andrew J. Rohm, and Tao Tony Gao (2009), “Factors
Reality Reshapes Mobile Marketing And Consumer-Brand Relationships,” Influencing Consumer Acceptance of Mobile Marketing: A Two-Country
Journal of Retailing and Consumer Services, 44, 11–23. Study of Youth Markets,” Journal of Interactive Marketing, 23, 4, 308–20.
Shah, Jaymeen R. and Mirza B. Murtaza (2005), “Effective Customer Sumioka, Hidenobu, Aya Nakae, Ryota Kanai, and Hiroshi Ishiguro (2013),
Relationship Management Through Web Services,” Journal of Computer “Huggable Communication Medium Decreases Cortisol Levels,” Scientific
Information Systems, 46, 1, 98–109. Reports, 3, 3034.
Sharma, Amit (2017), “How Retail can Thrive in a World Without Stores,” Sunaga, Tsutomu, Jaewoo Park, and Charles Spence (2016), “Effects of
Harvard Business Review, 2–5July 17. Retrieved from https://hbr.org/2017/ Lightness-Location Consumers' Purchase Decision-Making,” Psychology
07/how-retail-can-thrive-in-a-world-without-stores. & Marketing, 33, 934–50.
Shen, Hao and Jaideep Sengupta (2012), “If You Can't Grab It, It Won't Grab Turley, Lou W. and Ronald E. Milliman (2000), “Atmospheric Effects on
You: The Effect of Restricting The Dominant Hand on Target Evaluations,” Shopping Behavior: A Review of the Experimental Evidence,” Journal of
Journal of Experimental Social Psychology, 48, 2, 525–9. Business Research, 49, 2, 193–211.
———, Meng Zhang, and Aradhna Krishna (2016), “Computer Interfaces and Kerrebroeck, Van, Kim Willems Helena, and Malaika Brengman (2017),
the “Direct-Touch” Effect: Can iPads Increase the Choice of Hedonic “Touching the Void: Exploring Consumer Perspectives on Touch-Enabling
Food?” Journal of Marketing Research, 53, 5, 745–58. Technologies in Online Retailing,” International Journal of Retail &
Sherman, Elaine, Anil Mathur, and Ruth Belk Smith (1997), “Store Distribution Management, 45, 7/8, 892–909.
Environment and Consumer Purchase Behavior: Mediating Role of Van Noort, Guda, Hilde A.M. Voorveld, and Eva A. Van Reijmersdal (2012),
Consumer Emotions,” Psychology and Marketing, 14, 4, 361–78. “Interactivity in Brand Web Sites: Cognitive, Affective, and Behavioral
Shu, Suzanne B. and Joann Peck (2011), “Psychological Ownership and Responses Explained by Consumers' Online Flow Experience,” Journal of
Affective Reaction: Emotional Attachment Process Variables and The Interactive Marketing, 26, 4, 223–34.
Endowment Effect,” Journal of Consumer Psychology, 21, 4, 439–52. Varadarajan, Rajan, Raji Srinivasan, Gautham Gopal Vadakkepatt, Manjit S.
Simmons, W. Kyle, Alex Martin, and Lawrence W. Barsalou (2005), “Pictures Yadav, Paul A. Pavlou, Sandeep Krishnamurthy, and Tom Krause (2010),
of Appetizing Foods Activate Gustatory Cortices for Taste and Reward,” “Interactive Technologies and Retailing Strategy: A Review, Conceptual
Cerebral Cortex, 15, 1602–8. Framework and Future Research Directions,” Journal of Interactive
Song, Ji Hee and George M. Zinkhan (2008), “Determinants of Perceived Web Marketing, 24, 2, 96–110.
Site Interactivity,” Journal of Marketing, 72, 2, 99–113. Velasco, Carlos, Xiaoang Wan, Klemens Knoeferle, Xi Zhou, Alejandro
Spangenberg, Eric R., Ayn E. Crowley, and W. Pamela (1996), “Improving the Salgado-Montejo, and Charles Spence (2015), “Searching for Flavor Labels
Store Environment: Do Olfactory Cues Affect Evaluations and Behaviors?” in Food Products: The Influence of Color-Flavor Congruence and
Journal of Marketing, 60, 2, 67–80. Association Strength,” Frontiers in Psychology, 6, 301.
Spapé, Michiel M., Eve E. Hoggan, Giulio Jacucci, and Niklas Ravaja (2015), ———, Marianna Obrist, Olivia Petit, Kasun Karunanayaka, Adrian D. Cheok,
“The Meaning of the Virtual Midas Touch: An ERP Study in Economic and ——— (2016a), “Crossmodal Correspondences in the Context of
Decision Making,” Psychophysiology, 52, 3, 378–87. Digital Taste and Flavor Communications,” paper presented at CHI
Spence, Charles (2002), The ICI Report on the Secret of the Senses. London, Conference on Human Factors in Computing Systems, CA, USA, San Jose.
UK: The Communication Group. ———, Andy T. Woods, ———, Adrian D. Cheok, and Charles Spence
——— and Massimiliano Zampini (2006), “Auditory contributions to (2016b), “Crossmodal Correspondences Between Taste and Shape, and
multisensory product perception.,” Acta Acustica united with Acustica, Their Implications for Product Packaging: A Review,” Food Quality and
92, 1009–25. Preference, 52, 17–26.
——— and Alberto Gallace (2008), “Making Sense of Touch,” in Touch in ———, Marianna Obrist, ———, and Charles Spence (2018), “Multisensory
Museums: Policy and Practice in Object Handling. H. Chatterjee, editor. Technology for Flavor Augmentation: A Mini Review,” Frontiers in
Oxford, UK: Berg Publications, 21–40. Psychology, 9, 26.
——— (2011), “Crossmodal Correspondences: A Tutorial Review,” Attention, Vi, Chi T., Damien Ablart, Elia Gatti, Carlos Velasco, and Marianna Obrist
Perception, & Psychophysics, 73, 4, 971–95. (2017), “Not Just Seeing, But Also Feeling Art: Mid-Air Haptic
——— (2012), “Managing Sensory Expectations Concerning Products and Experiences Integrated in a Multisensory Art Exhibition,” International
Brands: Capitalizing on the Potential of Sound and Shape Symbolism,” Journal of Human-Computer Studies, 108, 1–14.
Journal of Consumer Psychology, 22, 1, 37–54. Vingerhoets, Guy (2014), “Contribution of the Posterior Parietal Cortex in
——— and Ophelia Deroy (2013), “Crossmodal Mental Imagery,” Multisen- Reaching, Grasping, and Using Objects and Tools,” Frontiers in
sory Imagery, New York, NY: Springer, 157–83. Psychology, 5, 151.
——— (2014), “Noise and its Impact on the Perception of Food and Drink,” Wan, X., A. Woods, C. Velasco, J. Deng, J. Youssef, and O. Deroy (2015), “On
Flavour, 3, 9. tasty colours and colourful tastes? Assessing, explaining, and utilizing
———, Nancy M. Puccinelli, Dhruv Grewal, and Anne L. Roggeveen (2014), crossmodal correspondences between colours and basic tastes,” Flavour, 4,
“Store Atmospherics: A Multisensory Perspective,” Psychology & Market- 23.
ing, 31, 7, 472–88. Wan, Xiaoang, Carlos Velasco, Charles Michel, Mu BingBing, Andy T.
———, Marianna Obrist, Carlos Velasco, and Nimesha Ranasinghe (2017), Woods, and Charles Spence (2014), “Does the Type of Receptacle
“Digitizing the Chemical Senses: Possibilities & Pitfalls,” International Influence the Crossmodal Association Between Colour And Flavour? A
Journal of Human-Computer Studies, 107, 62–74. Cross-Cultural Comparison,” Flavour, 3, 3.
Stanley, Andrew A. and Katherine J. Kuchenbecker (2011), “Design of Body- Wang, Xia, Chunling Yu, and Yujie Wei (2012), “Social Media Peer
Grounded Tactile Actuators for Playback of Human Physical Contact,” . Communication and Impacts on Purchase Intentions: A Consumer Social-
paper presented at World Haptics Conference, IEEE, 563–8. ization Framework,” Journal of Interactive Marketing, 26, 4, 198–208.
O. Petit et al. / Journal of Interactive Marketing 45 (2019) 42–61 61
Wilson, Margaret (2002), “Six Views of Embodied Cognition,” Psychonomic Yim, Mark Yi-Cheon, Shu-Chuan Chu, and Paul L. Sauer (2017), “Is
Bulletin & Review, 9, 4, 625–36. Augmented Reality Technology an Effective Tool for e-Commerce? An
Wober, Mallory (1991), “The Sensotype Hypothesis,” in The Varieties of Interactivity and Vividness Perspective,” Journal of Interactive Marketing,
Sensory Experience: A Sourcebook in the Anthropology of the Senses. D. 39, 89–103.
Howes, editor. Toronto: University of Toronto Press, 31–42. Zampini, Massimiliano and Charles Spence (2004), “The Role of Auditory
Woods, Andy T. and Charles Spence (2016), “Using Single Colors and Color Pairs Cues in Modulating the Perceived Crispness and Staleness of Potato Chips,”
to Communicate Basic Tastes,” i-Perception, 7, 4 2041669516658817. Journal of Sensory Studies, 19, 5, 347–63.