Experience Clip: Method for User Participation and

Evaluation of Mobile Concepts

Minna Isomursu, Kari Kuutti, Soili Väinämö
University of Oulu
P.O.BOX 3000
FIN-90014 UNIVERSITY OF OULU
{minna.isomursu; kari.kuutti; soili.vainamo}@oulu.fi

ABSTRACT mobile applications in versatile use situations. Because the

This paper describes experiences from using a field testing outdoor location itself (covering the pedestrian area in the Oulu
technique for collecting user experience information for city center, a dozen blocks or so) was an essential aspect of the
evaluating mobile applications used in everyday life. Our applications to be evaluated, there was no possibility to do
technique is based on the usage of mobile camera phones that testing in fixed locations. Our users were as real as possible -
are used for capturing video and audio during the use of the passers-by who got interested in the offer to experiment with the
mobile application. The users helped researchers in collecting service, published in a kiosk on the corner of the street. Using
user experience material by shooting the video clips themselves. questionnaires and interviews did not give enough detail, and
To our surprise they also started to participate actively by attempts to collect data by “shadowing” the users with a video
presenting "miniplays" in the clips to make their point clear. Our camera disturbed the actual situations so much that the gathered
results show that with this technique we can get richer emotional data was next to useless.
material and more versatile usage situations than with traditional The central idea of the technique is very simple: to avoid
observation methods, and additionally there is clearly a yet disturbance, we took pairs of users who were visiting the city
unexplored potential to develop a more systematic design center together, gave our application device to one and a
method around participation. separate mobile phone with a capability to take short video clips
to another, and asked the one with the phone to take video clips
Categories and Subject Descriptors when the other is using location-aware applications. This
H5.2 User interfaces, evaluation/methodology actually solved our problem of disturbance and gave us
information at the level of detail we wanted, but the video clips
we collected from our temporary collaborators contained, to our
General Terms surprise, also much more, and that is what we are going to
Design, Human Factors elaborate and reflect on in this paper.
We will proceed as follows. First, we describe the starting point,
Keywords our practical problem and the research setting. Then we present
User participation, mobile application, user experience, what we did, what happened with the clips, and also present
participatory design, use of video some results to our original questions. Finally, we discuss the
phenomenon we encountered from the viewpoint of
1. INTRODUCTION participatory design, and explore some directions for further
The purpose of this paper is to discuss a novel technique for research.
user-designer interaction. The technique was developed to solve
a practical need in evaluating mobile services, but it was found 1.1 The Challenge of Evaluating Mobile
that the technique has a strong participatory dimension. It also
brings together a number of different threads in research Applications
discussions on using video in participatory design situations. When HCI research moved from desktop applications towards
mobile devices and services, the practical problems related to
The initial problem that we tried to solve was how to collect doing evaluation in mobile environments rapidly become
useful fieldwork data for evaluation of location-based outdoor known. It is commonly accepted that data collection for
evaluation of mobile services and devices is a central challenge,
and that novel methods must be found for that. For example,
Permission to make digital or hard copies of all or part of this work for Newcomb et al. summarize their difficulties:” We found it very
personal or classroom use is granted without fee provided that copies are difficult to create a situated context to evaluate situated
not made or distributed for profit or commercial advantage and that copies interactions. Equally as difficult was capturing a user
bear this notice and the full citation on the first page. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prior
response...” [12], and Jones et al. see that “A key challenge will
specific permission and/or a fee. be to create an atmosphere that is more conductive to natural
patterns of use while being accessible to research and
Proceedings Participatory Design Conference 2004, Toronto, Canada. evaluation” [8]. Meanwhile, Palen and Salzman observe
Copyright 2004 ACM 1-58113-851-2/04/07…$5.00. “ecologically valid study of today’s mobile technologies is

83
challenging, because the conditions for technology use are
dynamic, varied, and difficult for investigators to directly
observe.” [13].
Thus, evaluating mobile applications is a challenging task: the
standard method of laboratory test loses the very idea of
mobility, and doing tests in real life contexts sets high
requirements for the data collection methods used. On the other
hand, a naturalistic setting provides evaluation possibilities not
available in the laboratory. Mobile applications are designed to
be used in a mobile context, so they should also be evaluated in
a mobile context. They are often used for supporting everyday
activities tightly woven into other activities taking place at the
same time. Therefore, for evaluating mobile applications, we
need to complement laboratory evaluations [1] with real life
field evaluations.
To address these requirements we need new methods for
collecting material for user evaluations. There have been a
number of attempts to overcome obvious difficulties generated
by mobility. Perhaps the easiest way is to try to continue the
laboratory practice in the field by "shadowing" users with a
video camera, but this is pretty obtrusive and also a heavy
method, more suitable for in-depth analysis than volume testing
with a number of users. Less obtrusive techniques which have
been tried out and reported include the use of clickstream data
logs [5], equipping a handheld device with a small camera [9],
and using the telephone or mobile device itself to collect voice
comments [13]. In long time use the Experience Sampling
method (ESM) [11] has been used for studying people in
naturalistic settings by reminding them every now and then via a
beeper device to fill in questionnaires probing their state of mind
and activities. Figure 1. Information kiosk.

In this paper we present yet another technique that eliminates the

influence of a researcher being present in the evaluation 2. USER EXPERIENCE EVALUATION
situation. The technique can be used in the mobile context. It Our goal was to evaluate the user experience evoked by the
encourages the users to freely express emotions, feelings and mobile applications in question. We wanted to identify factors
opinions about the application. that allow the design to positively influence the user experience.

1.2 Research Setting 2.1 User Experience

This research work has been carried out in a nationally funded By ‘user experience’ we mean the ‘totality’ of subjective
Rotuaari project. The project aims to evaluate context-aware experience of using a device or tool in a situation. As user
mobile applications in a real-world environment and with real experience is formed in a dynamic relationship between the user
end users. The research was carried out in downtown Oulu and and the device, the application and the usage environment, it
is named after its pedestrian area. The context-aware cannot be evaluated in a vacuum. [2]
applications evaluated were the following:
Location aware map. The user could see her location on
2.2 Problems with Evaluating User
the map and could search for landmarks and businesses Experience
located in the city area. We faced problems in capturing information about the factors
affecting the user experience in the field setting. We were
Context-sensitive advertisements. Advertisements were especially interested in the feelings, emotions and subjective
pulled up by the user device according to the context, opinions of the users when they used the applications. With
e.g. location and profile of the user. traditional methods this kind of information was hard to get.
The applications were used with PDAs that were loaned out We first used portable videocameras for capturing usage
without a fee to people visiting Rotuaari during the research sessions. The researcher followed the users, and recorded the
period of one month. To get a PDA the users were requested to whole usage sessions. Our experience showed that having a
fill in a simple user profile and agree that their actions would be videocamera in a public space such as the pedestrian area
logged for research purposes. The kiosk used for loaning out the restricts the mobility of the user, as the person with a
devices and instructing users is illustrated in Figure 1. videocamera cannot move very freely. Also, passers-by easily
stopped to stare at the strange-looking pair moving oddly. It was
also quite difficult to get ordinary users to participate in the
videotaped evaluation, so the test subjects needed to be recruited
beforehand.

84
A videotaped usage session provided an excellent overall picture not alter the environment in a way that would affect the
of the session under evaluation, but our experience showed that user experience.
videotaping in a public, mobile environment renders the usage
situation somewhat unnatural, hence potentially affecting the As mobile phones are very common in a modern city scene,
user experience. their usage in public places has become ubiquitous. People use
them frequently and their usage does not draw the attention of
We also tried out the method presented in [13] where users passers-by. Many new mobile phones have the capability to
observe each other by taking photographs with disposable record video and audio. These features are very easy to use
cameras and explain them with notes they write into a notebook. without extensive training or practice. Therefore, they can be
Our experience showed that the photos themselves did not give used in collecting user experience data without disturbing the
much information, as they typically showed a person standing in usage situation.
Rotuaari and staring at the PDA screen. However, some users
provided us with very interesting data in the notes they wrote in Our method explicitly encourages the users to participate in the
the notebook. The notes described the usage situation and the design process, as they are free to decide themselves what they
feelings related to the usage. We assume that taking a photo with want to record and what kinds of clips they will provide for the
a disposable camera indicates an important event and provokes research team. Users are naturally aware that the researchers are
the user to write a note in the notebook to explain the situation. their “audience” and researchers are interested in the messages
Therefore, even if the photographs themselves did not provide the users choose to express.
much extra information they presumably helped the users in the People often walk in a city center in small groups - with friends,
experience capturing process. colleagues or family members. We have exploited this by giving
However, not many users were able to produce notes detailed one person a PDA with the Rotuaari application and an
enough to provide us with rich information about user accompanying person a camera phone for recording the usage
experience. One reason for this might be that writing in a situations of the first person. In this way we also exploited a
notebook disturbs the normal mobile activities of the user, and natural human behavior of sharing experiences with friends. It is
requires her to sit down and start writing. Also, some people natural for a user to tell her friend about her feelings towards the
may find writing about their emotions and feelings difficult. device or the application during use.

We also applied traditional questionnaires for collecting

information related to the user experience. We used a
3.2 Rotuaari Experiment
We used two types of camera phones: Nokia models 3650 and
questionnaire where we applied the Emocards presented by
6220. Nokia supported our research by providing us with the
Desmet [4] for providing the users with a tool to describe their
equipment. Both phone models supported live video and audio
feelings about the application. However, we quickly found out
capturing. Camera phones with similar capabilities are available
that our research setting did not favor questionnaires. Pen-based
from other manufacturers, too, such as Sony-Ericsson and
questionnaire forms were difficult to use in practice because of
Samsung.
physical and practical constraints. Implementing an electronic
questionnaire would have required the users to use text input, We used the Experience Clip technique during three weekends
which proved to be difficult for our users who were usually not of the one-month field experiment period. During the
used to PDA devices with stylus input. experiment, a total amount of 36 people acted as observers with
camera phones. People often changed roles during usage, i.e. the
Because of the problems discussed above we decided to use a
PDA user became the observer and vice versa.
new approach for collecting user experience. We call our
approach the “Experience clip” technique. Our procedure during the field test was the following:
1. Give the users a PDA and a camera phone. Instruct the
3. EXPERIENCE CLIP TECHNIQUE PDA user on how to use the applications. Also, give
Because of the challenging research setting (i.e. real mobile instructions to the observer on how to record video clips.
usage environment and real end users), our requirements for the The instructions were the following:
user evaluation technique were complicated. We wanted
especially to collect information related to the user experience, Record as many clips as possible.
i.e. how the users experienced the usage situations, rather than Use the camera phone for capturing experiences
only how they managed to perform predefined tasks. Therefore, related to the usage: failures, success, surprise, joy,
we wanted to collect data from natural usage situations that anger, etc.
would not suffer from the influence of a research setting.
Aim at the user of the PDA, not at the PDA screen.
We showed how the video recording feature works.
3.1 Principles of the Technique
Our goal was to define a technique that would: We explained the technical constraints, such as the
best distance for shooting, keeping fingers away from
reduce or eliminate the influence of the researcher in the the lens, etc.
research situation,
2. Users spent time with the PDA and camera phones.
give users freedom to try out and explore the possibilities Because of the expected lifetime of the battery of the PDA,
of the system, we advised them to return within two hours.
support mobile usage situations, and 3. When the users returned with the devices, we asked them to
describe what they did, how they used the applications and

85
 how did the applications behave. We encouraged 4.3 Using a Researcher Instead of the User as
storytelling.
the Observer
4. Video clips were downloaded to a PC for further analysis We also used camera phones so that the users did not participate
and deleted from the phone’s memory. in recording video clips. Our researcher did the recording by
following the users when they used the Rotuaari system.
None of the users used the devices longer than one hour, so we
assume that the two-hour time limit did not have any effect on We soon realized that this affected the usage situation strongly.
our test setup. The usage sessions were short, and the users were not too
expressive about their feelings and emotions. When asked, they
4. EVALUATION OF THE TECHNIQUE explained that the presence of the researcher did not have any
Most of the videos shot complied well with our instructions and effect on the usage situation. They assured us that they were OK
provided us with good quality information about the usage with the researcher following them, and that it did not affect
situations. However, some of the material was irrelevant and their behavior. However, we could clearly see the difference in
provided little added value. Next, we will discuss the factors that the produced material. The video clips shot by the users
affected the success or failure of using the Experience Clip themselves were much richer in detail, contained more
technique. emotions, and conveyed a better feeling of real usage situations.
Also, where the researcher followed the user, in a problem
4.1 Instructions situation the user did not try to manage by herself, but
We found that to get high quality material, we needed to give an immediately asked for help from the researcher. When the users
instructional briefing to the camera phone user. If there was no observed each other, we got information about how the users
time to give instructions, the quality of the material was notably managed in problematic usage situations, and how they were
inferior. The observers needed to have explained to them what able to solve the problems faced during use.
kind of situations we wanted them to record. Also, as most of
them had not used camera phones previously, a short 4.4 Technical Problems
introduction to technical capabilities and restrictions was Capturing high quality audio in a city environment is extremely
necessary. challenging. Both our camera phone models had difficulties in
capturing low male voices, and the environmental noise, such as
4.2 Relevancy of Information street musicians and traffic light sounds, were also very
As the users were able to decide for themselves what to capture, disturbing. However, this problem was present also when we
we were concerned whether the material we would receive used the traditional videocamera.
would cover the whole spectrum of usage situations, or if it Both phones had a built-in limitation for the clip length. The
would be biased towards some specific types of uses. 6220 provided a possibility for longer clips (20 seconds), and it
Our results indicate that some users did not want their failures to stored the clip much faster than the other model, thus allowing
be recorded. They told the user of the camera phone to stop the user to start recording a new clip soon after finishing the
recording if they did not succeed in using the device. Some previous one. This can be seen in the length of the clips people
observers stopped filming, some continued recording. However, recorded. For example, the average total length of video clips
most users were quite happy to elaborate on their failures and recorded with the 6220 was 4 minutes and 8 seconds, whereas
negative experiences. Very negative experiences sometimes with 3650 people recorded an average of 2 minutes 22 seconds
resulted in shooting funny clips. For example, one user provided of video material. The maximum duration of material recorded
us a very realistic looking clip showing how he throws the with the 6220 was 10 minutes, whereas with the 3650 it was 3
device into the Baltic sea because he got so frustrated with it. minutes 37 seconds. Also, the quality of audio was better with
This may indicate that when the user experiences strong feelings the 6220. For these reasons, we found the 6220 to be better
towards the application and wants to express and communicate suited for our purposes.
her feelings to the designers, she no longer would only film what There are some downloadable applications available that remove
they were asked to film, but eventually would design and film the clip length limitation, and allow the user to record as much
their own ideas. This would expand the usage of video towards a video as the phone’s memory card can store. The technical
powerful multimedia ethnographic tool as described by quality of the resulting video is not as good as with the original
Goldman-Segall [6], and would provide the users means for applications provided by the terminal manufacturer. In addition,
participating in the design process through establishing a we experienced severe technical problems with downloadable
communication channel between the users and designers. video recording applications that resulted in losing video
Compared to other techniques we used, we found that with the material. Even though the recording application seemed to
Experience Clip technique we would get a much wider range of function properly, the application provided for viewing the
usage situations and emotional responses than with methods video clips refused to show the videos. Therefore, we preferred
where the researcher was involved in the usage situation. This using the original applications provided by the device
was probably caused by the fact that if the researcher followed manufacturer, even if the length of the video clip was restricted.
the users, they did not even try to use as many functions as they Some users had difficulties in keeping the viewfinder directed
did by themselves. Also, when the camera phone user was a correctly, especially when the PDA user was moving. With the
person well known to the user, it was very natural for the user to 3650 this was easier, as the screen used as a viewfinder is
explain their emotional responses and feelings towards the bigger. Keeping the viewfinder directed was especially difficult
application, and they could then be simultaneously captured. if the sun was shining straight into the camera. This made the
screen to turn white, although the quality of the video clip was

86
totally acceptable. Also, as the clip had to be shot from rather The resulting action-tables described the actions of the user,
close distance, the height differences of the PDA user and emotional signals expressed during the use (i.e. laughter or
observer sometimes caused the viewfinder to creep upwards or moans) and verbal discussions audible in the recorded clip. In
downwards. the analysis sessions, the paper-based written material was used
as a primary source of discussion, but the experience clips were
used at selected points to clarify or to emphasize selected topics.
4.5 Combining Clips
As the video clip length is very limited, the major problem in We did not have tools for connecting experience clips with the
analysing the material is how to perceive the whole chain of paper-based material, so the clips needed to be manually
actions that took place during use. For example, in one of the searched in the analysis session. This was naturally time
usage sessions the users had recorded several clips where they consuming and difficult. We are currently in the process of
tried to find a restaurant where they could have lunch, but were examining and evaluating tools that would support linking
having serious technical problems with the device. They had experience clips with written narratives.
shot several clips where they tried unsuccessfully to get the
system running. Then suddenly the next clip available showed
them having lunch in a restaurant. It was impossible to figure
5.8 Unexpected Participation Dimension of
out, if the system finally had helped them to find the restaurant, the Technique
or had they just chosen a restaurant near by without the help of Initially our idea in starting to experiment with the Experience
the system. This problem was tackled by using interview- Clip technique was just to overcome some pertinent problems in
initiated storytelling after the usage session. The users were the collection of use data in mobile environments. To our
requested to explain what they had been doing and how the surprise the experiments revealed a participatory dimension of
system behaved. With the help of the interviews we were able to the technique – a potential to use it in participatory design.
put the clips into context. Instead of staying within the predefined roles of user and data
collector, the user pairs took a much more active stance as
design collaborators, and created, acted out and recorded mini-
4.6 User Experiences Related to the dramas and mini comedies around use situations to make their
Experience Clip Technique point clear.
Users were usually willing and even enthusiastic in acting as
observers and using camera phones. In fact, some users were This is completely in line with the thread of thinking in
more interested in the camera phones than the applications we participatory design manifested by Buur et al. [3], that it is
were evaluating. Some users wanted to try out the features of the wasting video's potential to use it only as a record of use, ‘hard
camera phone, and this resulted in non-relevant clips, e.g. shots data’, when it can be used as a media in design and
of buildings, babies, bottoms of women walking in the communicating about design between stakeholders in the design
pedestrian area, etc. With a brief introduction, the users were process. It may be possible to develop a systematic approach to
usually capable of using the camera phones without difficulties. use experience clips in participatory design, but we have only
just started to explore those possibilities.
Many users seemed to have fun with observing each other. Most
usage situations recorded seemed very natural. In addition to
realistic usage situations, the users often created short
5. EXAMPLES OF RESULTS
performances. Especially frustrated users seemed to be likely to The rich material collected using the Experience Clip technique
express their feeling through performances. These performances was analysed by our research team. We give here examples of
also provided us with information about the emotional responses results we were able to extract from the material. The examples
of the users, even though they did not deal with real usage have been selected as representative of clips that include
situations. performances that are directed to the researchers. The
discussions between the users presented in the tables have been
Using a camera phone for recording usage situations in a city translated from Finnish by the authors. (It is a pity that it is not
setting did not cause notable disturbance to the environment. possible to show the delightful video clips themselves in the
People are so used to seeing mobile phones in a city scene that paper.)
passers-by did not pay much attention to our user groups.
However, we observed that using a location aware application
on a PDA in a mobile context in a public space was rather
5.1 Typical Usage Patterns
As the users were free to use the device and the related
dangerous. Our PDA users followed the screen of the device so
applications as they wanted, one of the things we were most
intensively that they forgot to watch their step. Luckily, our test
interested in was how the users actually used the system.
environment was mostly a pedestrian area, so we avoided
Experience clips provided us with a good tool for seeing how
serious injuries.
the users chose to use the system, and what features they found
most interesting. For example, our clips showed that users were
4.7 Analysis of Material first very excited about the positioning system (see Figure 2) and
Analysis of video material often includes transformation into they constantly followed the dot on the screen presenting their
paper-based descriptions for analysis purposes. With experience location on the map.
clips, we also needed to combine the clips with the interviews
done after the usage sessions. We created action-tables (similar The clips also provided the user a tool for expressing their prior
to the tables presented here in this paper) of usage sessions that expectations towards the system. Sometimes, there was a
described the actions of the user during one session in narrative mismatch between the prior expectations of the users and the
format. The experience clips were combined with the interview actual service provided by the system, and experience clips were
material to create a coherent narrative from the usage session. able to capture this. This provides designers with valuable

87
information about actual user needs. For example, the users
expected the mobile ads to provide them with special offers and Table 1. Experience clip describing response to positioning
great bargains, and were dissappointed when the advertisements PDA user Video Description
primarily included only facts about the shops and services. The shooter
disappointment caused users to create clips where they
expressed what kind of information they expected to receive “Does the dot Two young boys are
through mobile ads, and how the actual ads did not fulfill their move?” walking on the street.
needs.
“It moves now!” User is exited.
“Wow!” User stops.
“Check it PDA user moves
yourself!” towards video shooter
and shows device
screen.
“Show it to
me too.”
“It was there a User shows the route
moment ago, and on the screen and
now it is here.” sounds amazed.

The discussion captured in the experience clips illustrated by

Figure 3 and summarized in Table 2 shows an example of how
the users sent direct messages describing their emotional
Figure 2. Use of positioning system. responses to the research group.

5.2 Expressions of Emotions

The technique proved to be very effective in capturing
emotional experiences of the users. The clips revealed both
spontaneous emotional responses to the system as well as small
performances that were created by the users to express their
emotions.
The clip described in Table 1 illustrates spontaneous emotional
responses to the use of the positioning system. It also provides
an example of the social interaction between the user and
observer that has been captured by the experience clip. We
found that by creating usage situations where two users that
were familiar with each other could interact, the users were
better able to externalize and verbalize their experiences, than
when they used the system by themselves or with the presence
of the researcher, who was a stranger from the user point of
view.

Figure 3. A message to the designers

As the users participated in recording usage situations, they were
naturally aware of the research situation and knew that the
researchers would be an audience for the recorded clips.

88
 Table 2. Experience clip expressing frustration of the user system. For example, the experience clip described in Table 3
illustrates a situation where the user describes the difficulty of
PDA user Video Description
orientating and gives a direct proposal on how the directions
Shooter
could be shown on the map.
“Yes I can say this User looks at the
because we were device and looks Table 3. Experience clip describing difficulties in orientation
told to do so.” slightly irritated.
PDA user Video shooter Description
“Say it once User holds the device
again.” at arm’s length and “So which direction The user
examines it. do these streets explains to
actually go in? For someone else
“I am so mad. I do User walks the street example, if we now how the device
everything right but and looks angry. are on the corner of is used. He
this does not work.” Isokatu and looks at the
Saaristonkatu. Here device and
“You look User makes a face at should be something shows
awfully the device. like…” directions with
angry.” a stylus.
“This is so Video shooter laughs. “It is difficult to User laughs.
irritating! Damn… know in which
PDA user reads the
“…connecting…” message from screen. direction you
OKAY!” should go”
“Should there be
arrows pointing in
Frustration towards the device and applications was one of the the direction you
main triggers for creating dramatic performances (an example should go?”
can be seen in Figure 4).
“Yes, but there is no Video shooter
compass in the apparently also
device. But anyhow, concentrates on
that should be the the PDA screen,
idea.” as the picture
starts to wander

5.4 Technical Issues

The clips also provided us with information about the technical
reliability, accuracy and operation of the system.
For example, we had problems in our context aware
advertisement server that prohibited some users from receiving
advertisements. Most users were very curious about getting
mobile ads, so they ended up manipulating their profile and
changing their locations just in order to seek the advertisements.
This turned the usage of the device into a “seek-an-
advertisement” game instead of targeted advertising. A
combined set of remarks made by one user about receiving
mobile ads is presented in Table 4. The pair create a hyperbole-
loaded dramatic comedy around missing "push" advertisments.
The clip illustrates well how the users were able to cope when
the system did not technically operate as expected. It also gives
an example of a series of events that were captured during use in
serveral successive experience clips that together form a
narrative story for the researchers.
Figure 4. Frustrated user.

5.3 Usability Issues

As the users captured normal usage situations, the clips also
provided us with information about the usability of the system.
The users seemed to capture situations where they had
difficulties in understanding how the system works, or wanted to
propose a better solution for improving the usability of the

89
 Table 4. Excerpts of several experience clips from one user 6.1 Video as a Design Media
who did not receive any mobile ads Using video in recording use situations to be analyzed later, a
method borrowed from microsociology and anthropology, has
PDA user Video shooter Description already become an inseparable part of a variety of user-centered
design methods. During the last years, however, a discussion has
“We still haven’t User takes the
started that this kind of use means actually wasting the potential
received mobile ads. device out
video can have in design, and it is possible to develop more
from bag and
If the mobile ad active and efficient design methods by using video in a more
checks the
appears, what should participatory way.
screen.
I do?”
The major proponent of this kind of approach within
“I do not know how User tries to participatory design has been Jacob Buur, whose paper in PDC
to change the settings change her 2000 [3] was sort of a manifesto for the whole direction. In the
for receiving mobile profile in paper Buur et al. question the use of video only as “hard data”,
ads… We walk order to inherited from social science, where the emphasis is in the
around with the receive accountability of findings made by analytic researchers and
device, but no-one context based on that data. They then proceed by presenting their
sends us ads” sensitive ads. experiences in more active use of video as a design media, from
which they have four different examples: collaborative
“You would be a User looks designing and editing of a video documentary, co-authoring
good victim for puzzled and video materials, using video in collaborative sense-making with
commercialism” strokes her the help of a video “card game”, and using video as a reflective
hair. material by sharing tapes and negotiating stories. Another
“Really. I think that “But no-one sends example of this way of thinking is the work done by Helena
when we receive the ads” Karasti in her dissertation [10]. In her work, Karasti made video
first one we are going recordings at a radiology clinic and then prepared them as
to go there. No “video collages” that were used as discussion pieces in meetings
matter what it is between the future users of a radiology system and system
about.” designers. The same idea of active use of video as a reflective
medium is also a central part of the Change Laboratory method
“We’ll do that. developed at the Center for Activity Theory and Developmental
Should you check Work Research at the University of Helsinki (see [5]), in a larger
the ads again?” context than system design. There a set of video recordings
about problematic situations in everyday practice is used as a
“I can’t find any. User presses “mirror” to bring real life into situations where the further
Where was the ad different development of work is discussed.
window again…? buttons and
Nothing happens if I tries to seek
press the Info button. ads. She 6.2 Performances in Design
raises her Buur et al. observed, that some of their experiments had a
It’s really a pity if I voice in the "theatrical" character, which was considered highly useful for
don’t receive any end. design. The issue of the role and nature of performances in
ads.” design in general has been elaborated by Giulio Iacucci in
“I’ll check if I’ve Users walk several of his papers. For instance, Iacucci et al. [7] emphasize
received any ads.” around the the ephemerality and liveliness of performances when compared
city, sees to representations used in design that are static, and they identify
friends and three classes of situations where performances can relate to
try to “find” design. First, performances can support exploring and inventing
mobile ads. ideas, they can support representing and communicating a
scenario, and finally they can be useful in testing and
“Now the network User looks experimenting with an object. The authors further present a
doesn’t respond… disappointed number of examples from these three classes, and analyse them
We didn’t receive by means of three concepts. These concepts are fictional space,
any ads, even though imagination, and interactional creativity.
we wanted them!”
The fictional space is created by the participants by performing
and reacting to each other. It is not a substitution of reality, but a
representation of it, a space in-between, where rules and
conventions of reality can temporarily be surpassed so that
6. DISCUSSION fiction can take place.
Our experiences with the Experience Clip technique are related
to two active discussions within the participatory design Imagination, production of ideas, is related to the success of a
community, and it connects them in an interesting way. The two performance. Performance is succeessful when created ideas can
discussions are one on using video not only as data but as a be interpreted and reacted to by other participants, when they
design media, and another is on performances in design. change the fictional space as interpreted by some participant,

90
when ideas are inspired by the performance of physical actions to a particular user group, this may be a significant finding. The
in everyday contexts, or, when an idea produces an intervention obvious question and a task for further research is how this
in physical space that is inspiring and enlightening. participatory tendency can be best harnessed into the use of
design. For this, further and more systematic experiments are
Finally, interactional creativity refers to the emergent features of needed.
collective performances, where actions and "offers" by one
participant are "accepted" and reacted upon by others, and thus At least two further developments can be seen. One is to
novel features emerge when action unfolds. enlarge the sample: when camera phones are becoming
increasingly common, one can even envision large-scale
experiments based on the recording of video clips by using
6.3 Analysis of Experience Clips equipment that the interested people already own themselves.
Now our experience clips continue the discussion referenced
Another is to extend the participatory design community into a
above in an interesting way. Practically all the previous
larger crowd and intensify their participation and and to let users
examples mentioned in the literature where video material is co-
share their videoclips with others, comment on them and
produced (both Buur et al. and Iacucci et al. give a number of
combine them "to create a communal construction of meaning"
examples of this) have taken place so, that the researcher or
like Goldmann-Segal suggests ([6], p. 259), and thus perhaps
designer has been doing the recording, while users have been
create a "virtual community of users" (ibid.).
enacting a scenario or improvising with a mock-up device. Only
afterwards have users had the possibility to comment upon the
recording, or even to participate in the editing of a piece. 8. REFERENCES
Besides having a working system to be used, our setting differs [1] Beck, E., Christiansen, M., Kjeldskov, J., Kolve, N. and
from this by its use of a peer recording instead of a researcher or Stage, J. Experimental Evaluation of Techniques for
designer recording. Using Iacucci’s concepts, this generates a Usability Testing of Mobile Systems in a Laboratory
very different “fictional space” for the performance. The Setting. In Proceedings of OzCHI. (2003).
audience (researcher) both is there (because she has given the [2] Buchenau, M. and Fulton Suri, J. Experience Prototyping.
brief, and will watch the results) and is not there in person, only In Proceedings of the Designing Interactive Systems. ACM
as a projection in the mind of the performers. This, and the fact Press, (2000), 424-433.
that the two users, the recorder and the performer, are in full
control of the situations (what to record, how to behave) [3] Buur, J., Binder, T. and Brandt, E. Taking Video Beyond
apparently gives a much stronger sense of authorship to users 'Hard Data' in User Centered Design. In Proceedings of
than the normal situation, and feeds their imagination in a PDC 2000, (Palo Alto 2000), 21-29.
number of ways. Finally, the equal status between the recorder [4] Desmet, P., Overbeeke, C. and Tax, S. Designing Products
and performer apparently feeds the interactional creativity in the with Added Emotional Value; Development and
situation better than a more unequal situation between a Application of an Approach for Research Through Design.
researcher and a user: the perfomances were not monologs, but The Design Journal, (4(1) 2001), 32-47.
dialogs between the recorder and the performer - also in those
[5] Engeström, Y. Expansive Visibilization of work: An
cases where the recorder did not actually say anything.
Activity-Theoretical Perspective. In Proceedings of
With respect to earlier work, it is interesting to note that the Computer Supported Cooperative Work, (vol. 8 1999), 63-
technique seems to be situated somewhere in the middle ground 93
between Goldman-Segall's [6] very open participant use of [6] Goldman-Segall, R. Collaborative Virtual Communities:
video, and more designer-controlled approach by Buur et al [3], Using Learning Constellations, A Multimedia
who actually mildly criticize Goldman-Segall for a danger of Eghnographic Research Tool. In Sociomedia: Multimedia,
losing sight of a design project. Hypermedia, and the Social Construction of Knowledge
(Ed. Barrett E.). The MIT Press. (1992), 257-296.
7. CONCLUSION AND FURTHER [7] Iacucci, G. Iacucci, C. and Kuutti, K. Imaging and
RESEARCH Experiencing in Design – the Role of Performances. In
Our experience shows that the Experience Clip technique can Proceedings of the 2nd Nordic Conference on Human –
provide rich data about the emotions, feelings and experiences Computer Interaction, (Aarhus September 2002).
evoked by a mobile application, if certain prerequisites are [8] Jones, M.L.W., Rieger, R.H., Treadwell, P. and Gay, G.K
fulfilled. First, the users need to be instructed on what they are Live from the stacks: User Feedback on Mobile Computers
expected to capture, and how they should do it. Second, the and Wireless Tools for Library Patrons. Digital Libraries,
users need to be motivated and willing to participate. The (2000), 95-102.
technique can be recommended for capturing data from mobile
usage situations, and for evaluating applications outside the [9] Kaikkonen, A. and Roto, V. Navigating in a Mobile
office environment. Thus, the Experience Clip technique can be XHTML Application. CHI Letters (vol. No. 5, Issue No. 1
useful as a use data collection technique for mobile devices and 2003), 329-336.
applications, [10] Karasti, H. Increasing sensitivity towards everyday work
practice in system design. Acta Universitatis Ouluensis,
Additionally, the technique creates a situation where users can
University of Oulu (available also from
easily step beyond being only collectors of data for design into a
http://herkules.oulu.fi/isbn9514259556/ ), 2001.
more active role, to invent potential new uses and suggest
improvements. Because the difficulty of organizing genuine [11] Larson, R. and Csikszentmihalyi, M. The Experience
user participation is one of the actual problems in the Sampling Method. In Naturalistic approaches to studying
development of generic mobile services not accurately targeted social interaction: New Directions for Methodology of

91
 Social and Behavioural Science (Ed. Reis, H.T.) San [14] Pöysä, J., Mäkitalo, K. and Häkkinen. P. A Participant
Francisco. Jossey Bass. 1983. Experience Method for Illustrating Individuals’
[12] Newcomb, E., Pahsley, T. and Stasko, J. Mobile Experiences in the Course of and Evolving Virtual
Computing in the Retail Arena. CHI Letters (vol. 5 No. 1. Learning Community. Designing for Change, (2003), 451-
2003), 337-344. 460

[13] Palen, L. and Salzman, M. Voice-Mail Diary Studies for [15] Waterson, S., Landay, J.A. and Matthews, T. In the Lab
Naturalistic Data Capture under Mobile Conditions. In: In and Out in the Wild: Remote Web Usability Testing for
Proceedings of ACM CSCW'02, (2002), 87-95. Mobile Devices. In CHI2002 Short Paper, 2002.

92