Cold Calling

Journal of Management Education
Impact of Cold-Calling XX(X) 1–37

© The Author(s) 2012
Reprints and permission:
on Student Voluntary sagepub.com/journalsPermissions.nav
DOI: 10.1177/1052562912446067
Participation http://jme.sagepub.com
Elise J. Dallimore1, Julie H. Hertenstein1, and

Marjorie B. Platt1
Abstract
Classroom discussion is perhaps the most frequently used “active learning”
strategy. However, instructors are often concerned about students who are
less inclined to participate voluntarily.They worry that students not involved
in the discussion might have lower quality learning experiences. Although
instructors might consider whether to call on a student whose hand is not
raised (“cold-call”), some instructors resist cold-calling fearing that the stu-
dent will feel uncomfortable. This study examines the impact of cold-calling
on students’ voluntary participation in class discussions and their comfort
participating in discussions. The results demonstrate that significantly more
students answer questions voluntarily in classes with high cold-calling, and
that the number of students voluntarily answering questions in high cold-
calling classes increases over time. Furthermore, students in classes with
high cold-calling answer more voluntary questions than those in classes with
low cold-calling; this also increases over time. Finally, in classes with high
cold-calling, students’ comfort participating in class discussions increases
while in classes with low cold-calling, students’ comfort participating does
not change. Research findings show that cold-calling can be done fairly ex-
tensively without making students uncomfortable. Thus, the research re-
ported here provides support for using this instructional strategy to engage
more students to participate in discussions.
1
Northeastern University, Boston, MA, USA
Corresponding Author:
Julie H. Hertenstein, College of Business Administration, Northeastern University, 404 Hayden
Hall, Boston, MA 02115-5000, USA
Email: [email protected]
2 Journal of Management Education XX(X)
Keywords
cold-calling, student voluntary participation, students’ comfort participating,
class participation, class discussion, case discussion
For more than 20 years, educational philosophers and practitioners have

encouraged instructors to use strategies designed to actively engage students
in the teaching–learning process (Bonwell & Eisen, 1991; Nyquist & Wulff,
1990). Classroom discussion is perhaps the most frequently used and more
often embraced “active learning” strategy. However, when leading classroom
discussions, instructors are often concerned about students who are less
inclined to participate voluntarily. They worry that if discussion engages stu-
dents in their own learning—and, as suggested by some, improves student
performance and satisfaction (Bonwell & Eisen, 1991; Dallimore, Hertenstein,
& Platt, 2010)—then students not actively involved in the classroom discus-
sion might have a lower quality learning experience.
One approach to increase the number of students engaged in classroom
discussion is to cold-call students; we use the term cold-call to refer to any
instance in which a teacher calls on a student whose hand is not raised.
Nevertheless, despite the fact that cold-calling might engage more students
in the discussion, some instructors resist the idea of cold-calling as they fear
that a student who is cold-called will feel uncomfortable, or worse, humili-
ated (Dallimore, Hertenstein, & Platt, 2005). Student responses, however,
suggest that this may not necessarily be the case (Dallimore, Hertenstein, &
Platt, 2006; Rosmarin, 1987; Souza, Dallimore, Aoki, & Pilling, 2010).
This study examines the impact of cold-calling on students’ voluntary par-
ticipation in class discussions and their comfort participating in discussions in a
required business course. The primary results indicate that cold-calling is effec-
tive at increasing the number of students who answer questions voluntarily; fur-
thermore, in classes with high cold-calling, voluntary participation increases
over time. Finally, comfort participating in class discussion increases in classes
with high cold-calling; it does not change in classes with low cold-calling.
Our results make important contributions to the literature on discussion
teaching in general and cold-calling in particular. First, we demonstrate that
significantly more students answer questions voluntarily in classes with
high cold-calling in contrast to those with low cold-calling, and the number
of students voluntarily answering questions increases over time. Furthermore,
students in classes with high cold-calling answer more voluntary questions
than those in classes with low cold-calling, and this also increases over time.
These findings suggest greater student engagement in classes with high
cold-calling as more and more students participate voluntarily and more fre-
quently. Finally, in classes with high cold-calling, students become more
Dallimore et al. 3
comfortable participating in discussions, whereas in classes with low cold-

calling students’ comfort participating in discussions does not change. These
findings are consistent with the development of communication skills more
generally; that is, the more a student practices participating in class discus-
sions (even if via cold-calling), the more skilled the student becomes, and
the more comfortable he or she becomes when using this skill. Thus, the
findings should encourage instructors to consider using cold-calling tech-
niques to develop discussion skills that will enable students to become com-
fortable participating voluntarily in class discussions.
We begin by reviewing the literature on class discussion as a means for
engaging students in their own learning, and the literature on cold-calling as
a means to engage students in class discussion. Next, we present our research
hypotheses, methodology, and results followed by our discussion and impli-
cations. After we acknowledge the limitations of this study, and make sug-
gestions about future research, we provide our concluding remarks.
Literature Review
Active learning—which encourages student collaboration in the teaching–
learning process—has long been encouraged as an educational strategy
(Bonwell & Eisen, 1991; Nyquist & Wulff, 1990). Discussion teaching is
one of the most widely used forms of active learning. Discussion provides
the opportunity for students to practice and develop essential skills such as
organizing concepts, formulating arguments, evaluating evidence, and
responding thoughtfully and critically (Davis, 2009) and elicits higher level
reflective thinking (Ewens, 2000). Through participation in discussion, stu-
dents are better able to retain information (Ewens, 2000) and to learn more
(Weaver & Qi, 2005). Several explanations have been proposed for why
students who participate in discussion learn more. Arbaugh and Benbunan-
Fich (2006) suggest that more learning results because it is easier to inter-
nalize explanations from knowledgeable peers and because disagreements/
misunderstandings can be resolved during group discussions. This explana-
tion is consistent with Hertenstein’s (1991) earlier assertion that discussions
provide students the opportunity to learn “both through their own active
participation and through the contributions of others” (p. 175).
In the field of business, references in the literature to class discussion
often focus on case discussion as the case method is one of the most common
and important pedagogical techniques used to teach business (Greenhalgh,
2007; Rollag, 2010). Indeed, it has been said that cases have become “ubiq-
uitous” in management education (Schmidt-Wilk, 2010, p. 492); case discus-
sions provide varied benefits. Gilmore and Schall (1996) find that case
discussions are preferred to lectures when training future business leaders

because they serve to enhance students’ problem-solving skills. Liang and
Wang (2004) indicate that cases “bring ‘a chunk of reality’ into the classroom
so that students can learn how to solve complex problems in the real world”
(p. 409). C. R. Christensen and Hansen (1987) further suggest that this peda-
gogy benefits students by enabling them to develop their own framework for
dealing with business problems and by creating an environment where intel-
lectual inquiry, risk taking, and new learning occur. C. M. Christensen and
Carlile (2009) indicate that by drawing lessons from experiences with case
discussions, students will be prepared to be discerning consumers of theory;
this will help them to instinctively know whether they can trust the recom-
mendations of a consultant or subordinate in a given circumstance.
More specifically, in accounting, case discussions are considered to be
effective in part because they demonstrate how accounting contributes to the
development and communication of information in organizations (Bruns,
2006). Case discussions are especially suitable for management accounting,
which focuses on decisions managers must make to achieve their organiza-
tional objectives and the information required to support these management
decisions. Case discussions provide students the opportunity to analyze var-
ied alternatives for selecting, preparing, and communicating information
and to choose those most appropriate for the company’s unique situation and
its strategic goals.
As any classroom teacher knows, not all students are equally likely to partici-
pate voluntarily. Although some reticence may be because of learning styles,1
Dallimore et al. (2010) suggest a broad-based relationship between participation
and learning. That study shows that the frequency of a student’s participation is
positively associated with learning and that the participation of other students
increases learning. Thus, if students do not participate in class discussions, they
may not learn as much, and other students in the class may not learn as much.
Many different approaches have been suggested to increase the number of
students who participate, especially voluntarily. It has been noted that it is
important for instructors to set clear expectations about participation (Scollon
& Bau, 1981), to provide participation feedback and evaluation rubrics (Arter
& McTighe, 2001; Stevens & Levi, 2005), and to encourage all students to
participate (Davis, 2009). Others have noted specific techniques such as study
questions and response logs (Fishman, 1997) or assigning roles in discussions
(Smith & Smith, 1994). More recently, there have been references to using
technology to increase student participation in discussion—including clickers,
online discussions, or even online course delivery (Arbaugh, 2000; Brower,
2003; Bump, 1990; Carnaghan & Webb, 2007; Cunningham, 2008; Ellis,
Dallimore et al. 5
Goodyear, Prosser, & O’Hara, 2006).2 Although any of these approaches may
help involve some students who are reluctant participators, they do not resolve
this problem entirely.
A student’s ability to participate in class discussions is an acquired skill,
which, like other communication skills such as making formal presentations,
develops over time. Both academic and practical sources suggest that with such
skills, the more students practice, the better they get, and the more comfortable
they become (“10 Tips for Public Speaking,” n.d.; Dallimore, Hertenstein, &
Platt, 2008; “The Habit of Courage,” n.d.; Sprague & Stuart, 2003). Students
explain that they are more comfortable participating when they participate
more and when other students participate more (Souza et al., 2010). One reason
students’ comfort is enhanced when engaged in discussion with other students
is because they relate to one another as peers. C. R. Christensen (1991) explains,
Students also tend to share the language system of their generation, a

common idiom of “go” and “no go” words and relevant metaphors. . . .
This, plus their knowledge of fellow students brings them swiftly to the
core of effective communication, speaking to not at one another.
Equally important, it is simpler and less threatening for participants to
check and recheck each other’s meaning than for the instructor to do
so. (p. 109)
Although some students may have developed sufficient class discussion

skills, others may not. Those who have not may be knowledgeable and well
prepared for the class, yet they may be unwilling or unable to volunteer
because they lack well-developed discussion skills. Since comfort participat-
ing in class discussions has been shown to be associated with increased fre-
quency of participation (Dallimore et al., 2010), instructors may need to bring
these less willing students into the discussion—even if not voluntarily—so
that they can develop the skills to participate and can become comfortable
volunteering.
Cold-Calling: A Technique to Increase Class Participation

There is an emerging literature on cold-calling which examines the relation-
ship between cold-calling by instructors and students’ participation. First,
when students were asked to respond to questions about what instructors do
or say that (a) increases the quality of student participation and (b) increases
the effectiveness of the discussion, a key theme in their responses to both
questions was cold-calling (Dallimore, Hertenstein, & Platt, 2004).
Although it may not be obvious why calling on a student whose hand is not
raised might increase student participation quality and discussion effective-
ness, a study by Dallimore et al. (2006) provides some insight. The study
examines one graduate class taught by an instructor who cold-called fre-
quently and graded participation. The primary findings were that not only did
students’ self-reported participation frequency increase, but their self-reported
preparation increased as well. If cold-calling is associated with increased
preparation, then preparation may enhance participation quality and discus-
sion effectiveness. This link between cold-calling and preparation is noted in
an earlier edited book about case teaching (Rosmarin, 1987).
Despite the apparent potential for cold-calling to enhance participation qual-
ity, discussion effectiveness, frequency of participation, and preparation, some
instructors are hesitant to call on a student whose hand is not raised. These
instructors worry that cold-calling might make students feel “humiliated” or
that they were “victims” (Dallimore et al., 2005, p. 24). Experienced instructors
further suggest that because cold-calling would make students uncomfortable,
they would be less likely to volunteer (Dallimore et al., 2006).
However, in the limited research available where data are gathered from
students rather than instructors, the evidence suggests that cold-calling might
not have a negative impact on student comfort participating. Dallimore et al.
(2006) report that average student comfort participating in class did not
decrease between pre- and postcourse surveys in one graduate course taught
by an instructor who cold-called frequently. They further note that increased
preparation and increased participation frequency are associated with
increased comfort participating. Souza et al. (2010) report results from a study
where teachers in some courses cold-called, whereas teachers in other courses
did not cold-call. Using pre- and postcourse surveys, they report that comfort
participating increased significantly in courses with and without cold-calling.3
Thus, student perceptions about the effects of cold-calling on student comfort
participating seem to contradict the views of instructors.4
However, the earlier studies have methodological limitations, which also
raise concerns about some of their findings. For example, student participa-
tion frequency was measured based solely on student self-reported fre-
quency. It is possible that at the end of the course, students had a positive
perception of their participation and perceived that they participated more
than they actually did. In addition, the earlier studies do not distinguish vol-
untary participation from participation resulting from cold-calls. Thus, the
perception of increased frequency may result solely from cold-calling, and
students’ voluntary participation may not change.
Dallimore et al. 7
Furthermore, earlier studies have no measure of the number of students

who actually participated (as contrasted with the frequency of a student’s
participation), either voluntarily or via cold-call. The earlier studies also rest
on the instructors’ assertions that they did or did not cold-call. That is, these
earlier studies do not objectively measure cold-calling, and there is no evi-
dence on how frequently it occurs or how much it varies between instructors.
Although it may have been reasonable for these exploratory, preliminary
studies to use student perceptions of participation frequency and instructor
assertions about the use of cold-calling, there is a need for more comprehen-
sive studies with more objective measures of participation and cold-calling.
Finally, the earlier studies gathered data by survey at two points in time:
precourse and postcourse. If a student’s class discussion skill develops over
time, data gathered at additional points in time may enable us to more fully
understand the discussion skill development process.
This study is designed to extend our understanding of the effects of cold-
calling by addressing these limitations. It extends earlier studies by using
observational data from independent observers to measure students’ class
participation. The observational data provide an objective means to deter-
mine not only the total number of students participating in the discussion
but also the total number of students who volunteered or were cold-called.
Furthermore, instead of relying on student perceptions of their class partici-
pation frequency, this study uses actual counts of student participation in
the discussion and measures not only the frequency of their participation as
in previous studies but also identifies whether that participation was volun-
tary or resulted from a cold-call. This latter measure—the total number of
cold-call questions asked—provides an objective measure of the degree to
which instructors use cold-calling instead of relying on instructor asser-
tions that they did indeed cold-call their students. Finally, the observational
data provide additional data points, which, when combined with precourse
and postcourse surveys, can more clearly reveal the class discussion-skill
development process.
Given the recent study linking student participation frequency and student
learning (Dallimore et al., 2010), it is especially important to examine the
relationship between cold-calling, student participation frequency, and student
comfort more systematically and in greater depth. In particular, we need to
examine whether cold-calling is related to changes in voluntary participation
frequency and comfort over time since we have suggested that discussion
skills develop over time. If instructors better understand how cold-calling
affects the frequency of participation—especially volunteering—in their
classroom, and if they have better evidence on the relationship between
cold-calling and student comfort in class participation, they will be better able
to decide how much cold-calling to use.
Research Hypotheses
One factor that characterizes student participation in class discussions is the
number of students who participate in the class. For example, in a 40-stu-
dent class, participation would be considered to be greater if 30 students
participate than if only 10 students participate. In heavy cold-calling envi-
ronments, some students may be brought into the discussion for the first
time. Given this first opportunity to exercise and develop their class discus-
sion skills, these individuals may find themselves more willing to volunteer
to participate in subsequent class discussions. Through cold-calling, par-
ticipation opportunities may be offered to more students throughout the
course, so we expect the percentage of students who voluntarily answer
questions to increase. Thus, Hypotheses 1a and 1b are as follows:
Hypothesis 1a: Cold-calling positively affects the percentage of stu-

dents who voluntarily answer questions.
Hypothesis 1b: The percentage of students who answer questions vol-
untarily increases over time in sections with high cold-calling, but
does not change in sections with low cold-calling.
Another characteristic of student participation in class discussions is how often

a student participates in a class session. For example, participation would be con-
sidered to be greater if students participate three times versus if they participate
once. Analysis of data from graduate students in a heavy cold-calling environment
suggests that students’ self-reported frequency of participation increases despite
instructor concerns that cold-calling might make students uncomfortable and
hence less willing to participate (Dallimore et al., 2006). In a heavy cold-calling
environment, some students may be given more opportunities to participate than
they typically have. If these additional opportunities to participate in the discus-
sion contribute to the development of students’ class discussion skills, then stu-
dents may become increasingly willing to volunteer to answer questions in the
class discussion.5 As a result, we expect the frequency of students’ voluntary
participation to be greater when cold-calling is used extensively, and we expect it
to increase over time. Thus, Hypotheses 2a and 2b are as follows:
Hypothesis 2a: Cold-calling positively affects how frequently students
participate voluntarily.
Dallimore et al. 9
Hypothesis 2b: The frequency that students participate voluntarily

increases over time in sections with high cold-calling but does not
change in sections with low cold-calling.
A student’s overall frequency of participation includes the number of

times each participates voluntarily and the number of times each is cold-
called. If cold-calling positively affects how frequently students participate
voluntarily as hypothesized above, then cold-calling will positively affect the
overall frequency of their participation. That is, when cold-calling is fre-
quent, students will be cold-called more and they will volunteer more. Thus,
Hypothesis 2c is as follows:
Hypothesis 2c: Cold-calling positively affects the overall (voluntary

plus cold-called) frequency of students’ participation.
Despite instructor concerns that cold-calling will make students uncom-

fortable, there is some evidence that this may not be the case. Dallimore et al.
(2006) found that student comfort with class discussion in a high cold-call
environment did not change over the course of the term. More recently,
Souza et al. (2010) found that student comfort with class discussion
increased in both control (no cold-calling) and treatment (some cold-calling)
groups; however, these researchers did not directly compare the increased
ratings for the two groups. Because of design or methodological issues,
these previous studies have not provided definitive evidence on the effect
of cold-calling on the change in student comfort with class discussion over
the course of the term. Looking at factors related to student comfort with
class discussion, Dallimore et al. (2010) found that increased participation
in class discussion led to increased comfort. Because Hypotheses 1 and 2
posit a positive relationship between cold-calling and voluntary participa-
tion over time, we expect that student comfort with class discussion should
increase over time in high cold-calling sections whereas student comfort in
low cold-calling sections should not change. Thus, Hypothesis 3 is as
follows:
Hypothesis 3: Student comfort participating in class discussion

increases in sections with high cold-calling but does not change in
sections with low cold-calling.
Method
Research Design
The research design involved gathering data at various times in 16 sections
of a single course in which 632 students were enrolled. Data were gathered
using two surveys administered to students and observations of two class
discussions for each section.
Course
This research was conducted in the undergraduate management accounting
course required for the BS in Business Administration degree at a large,
private university. This course is taught on-ground, not online. The course
was selected because it has multiple sections taught by multiple instructors
each year, and a common syllabus and final exam are used across sections.
Although the sections are not identical because of individual differences in
instructional style between instructors, all sections share common peda-
gogical elements such as syllabus, textbook, slides for lectures, cases, final
exam, and so on. The pedagogy includes case discussions, written case
analyses, small-group problem solving, lecture, team projects, and student
presentations.6
Data were gathered from 16 sections of this course taught in two consecu-
tive fall semesters. The course met twice a week in 100-minute sessions for a
total of about 25 sessions. There were approximately 40 students in each sec-
tion and attendance was required.
Data
Two surveys were used to gather data on student opinions and perceptions
about class discussions; in addition, each section was observed twice during
the course. Thus, data were gathered at four times during the course as
shown in Figure 1.
Surveys. The purpose of the precourse survey was to establish a baseline prior
to the course of students’ attitudes and behaviors related to class participation.
To avoid biasing students, it did not mention cold-calling or calling on students
whose hands were not raised. The survey stated, “We are conducting a study on
class discussion and student learning to help faculty understand approaches that
lead to student learning so they may become more effective teachers.”
The postcourse survey focused on participation frequency and comfort par-
ticipating in this course. Although the postcourse survey did not mention
Dallimore et al. 11
Pre-course Post-course
Observa!on 1 Observa!on 2
Survey Survey
(Weeks 6-7) (Weeks 9-10)
(Weeks 1-2) (Weeks 13-14)
Figure 1. Time line for survey and observation data (week in course for data
collection in parentheses)
cold-calling per se, it asked whether the professor had called on students
whose hands were not raised in this course, and, if so, the effect on the overall
discussion and the respondent’s preparation.
The appendix lists precourse and postcourse questions analyzed in this
study. Students responded to these questions using a 7-point Likert-type
scale. They also provided their gender.
Observations. Days when case discussions were held were selected for
observation to maximize the periods of discussion observed. The observers
were graduate students employed as research assistants. There was one
observer for each case discussion class.
Training sessions were held to teach observers to categorize questions
and record data. The initial training session was followed by multiple prac-
tice sessions. Each practice was followed by debriefing and discussion to
clarify the coding process. Practice sessions were repeated until consistency
in coding was achieved across observers. Following the training session,
observers recorded data in actual classes not in the study and discussed their
results with researchers before conducting research observations.
Observers were told that they were assisting in research about discussion
teaching. Observers did not have detailed information about the research
project; specifically, researchers did not discuss the hypotheses with the
observers.
At the beginning of each case discussion observed, the observer recorded
the gender of each student and her location (seat) in the classroom on a seat-
ing chart diagram. For each question the instructor asked during the discus-
sion, the observer recorded which student responded and whether the
student volunteered or was cold-called.
The students observed in this study were enrolled in one of the first
courses where they encounter case discussions; thus, instructors need to
actively facilitate the discussion to engage students and help the discussion
to progress. Questions related to all levels of Bloom’s taxonomy (Anderson
& Krathwohl, 2001; Bloom, 1956) were used by instructors in this study to
facilitate the discussion.7
Students
Students enrolled in this course are typically sophomores. Class rosters
issued 1 week prior to the start of the course indicated that 632 students
were enrolled in the 16 sections of the course; rosters at the end of the
course indicated that 598 students were enrolled. To ensure confidentiality,
students were not asked for their names on the surveys; however, they were
asked for the last four digits of their nine-digit student identification number
to enable precourse surveys and postcourse surveys to be matched for
analysis purposes. Precourse surveys were completed by 581 students and
postcourse surveys by 541. A total of 359 students provided accurate iden-
tification information on both the precourse and postcourse surveys allow-
ing surveys to be matched, consisting of 46% (166) women, and 54% (192)
men.8 During the first observation, 578 students were present, for the sec-
ond, 559. The proportions of men and women in the two observations were
approximately the same as for the surveys.
Instructors
The 16 sections of this course were taught by seven full-time instructors. Two
were tenured faculty, four were tenure track but nontenured faculty, and one
was full-time but not tenure track; two were women and five were men. The
Informed Consent form signed by each instructor indicated that they were
participating “in a research study on class discussion and student learning . . .
designed to identify factors that affect student learning during class discus-
sions.” Instructors were not informed about the hypotheses.
All instructors in this research project were considered effective teachers.
The teaching-effectiveness ratings for all sections in this study averaged
4.51 out of a maximum possible score of 5.0.9 Furthermore, the ratings for
“treats students with respect” for all sections in this study averaged 4.64 out
of 5.010; thus, all instructors in this study were considered to provide a sup-
portive learning environment.
No constraints were imposed on these instructors; each of them used their
natural instructional style, including whether, or how much, they cold-called
students. Variations between instructors were because of individual differences
in instructional styles.11
Dallimore et al. 13
Data Analysis
Dependent Variables: Observation Data
Percentage student participation in class discussion. During each observa-
tion period, the behavior of individual students (based on seat location
during the class, not name or student identification number) was observed
and recorded, as described above. Based on these recorded observations, a
student who answered at least one volunteer question was given a value of
1 on the “answered a voluntary question” variable; otherwise, the student
was given a 0. Likewise, a student who answered at least one cold-called
question was coded as a 1 on the “answered a cold-called question” vari-
able; otherwise the student was coded as a 0. The percent of students pres-
ent who answered either volunteer or cold-called questions was calculated
for each section. Together, these variables measured the extent to which
there was broad student participation in the class discussion in a particular
class section.
Participation frequency per student. Because student behavior was recorded
with respect to type and number of questions answered, a measure of fre-
quency of class participation was obtained by counting the number of volun-
teer questions answered by each student present and the number of
cold-called questions answered by each student. These metrics focus on par-
ticipation frequency at the individual student level of analysis.
Dependent Variables: Survey Data. As discussed above, two surveys were

administered to assess student opinions and perceptions of class discussion.
The survey variables used as dependent variables in this analysis included
measures of participation frequency in class discussion and measures of com-
fort participating in class discussion.
Independent Variables
Cold-call environment. The primary focus of this study is to compare stu-
dent behavior and perceptions in differing cold-call environments. Instruc-
tors used their own pedagogical style, including whether or how much they
cold-called students; we did not actively manipulate this variable. In one
section no cold-calling was used; in all other sections instructors used some
combination of cold-calling and voluntary questions. Because instructors
used their own pedagogical style, we used the observation data to classify
sections into high cold-call (Hi CC) versus low cold-call (Lo CC) environ-
ments. Based on observation of the first case discussion for each section, the
mean percentage of students cold-called was calculated for each section; it

ranged from .00 to .84 with an overall mean for all sections of .26. The over-
all mean percentage of students cold-called was used to separate sections
into Hi CC versus Lo CC groups. The mean percentage of students cold-
called in Hi CC sections was .61, ranging from .33 to .84 versus a mean of
.10 of students cold-called in Lo CC sections, ranging from .00 to .24. This
grouping rule resulted in 5 sections with 196 students rated as Hi CC and 11
sections, with 486 students, as Lo CC. To exclude alternative explanations
for differences between Hi CC and Lo CC sections, we compared mean
teaching effectiveness ratings, “treats students with respect” ratings, and
years of teaching experience for the instructors between the two groups. We
found no significant difference between Hi CC and Lo CC sections for any
of these variables (p = .68 for teaching effectiveness, p = .24 for “treats stu-
dents with respect,” p = .42 for years of experience).
Analysis of percentage of students cold-called in both observation periods
validates this grouping methodology. From the observation data, the percent-
age of students present who answered cold-called questions was analyzed
using a full factorial analysis of variance (ANOVA) with cold-call environ-
ment and observation period as the two fixed independent variables. The
results reveal a significant main effect due to cold-call environment, F(1,
1134) = 267.81, p = .000, with the Hi CC sections having a significantly
higher proportion of students who were cold-called (.45) than the Lo CC sec-
tions (.02) across both observation periods. Neither the main effect of obser-
vation period nor the two-way interaction effect was statistically significant.
Furthermore, student survey results also validate this grouping methodology.
In the second year of the study, the postcourse survey asked students, “How
often did your professor call on students whose hands were not raised?”
based on a Likert-type scale from 1 = very little (not more than once in a
class) to 7 = a lot (seven or more times per class).12 A t test comparing the
means between Hi CC and Lo CC sections revealed a statistically significant
difference between cold-call environments, t(189) = −6.36, p = .000; the
mean of the Hi CC sections (4.93) was significantly higher than the Lo CC
sections (3.45). Thus, students perceived a significant difference in their
instructor’s classroom behavior with respect to cold-calling consistent with
the difference observed in the percentage of students cold-called during the
first observation period.
Time of observation. In each section, two case discussions were observed
and behavior recorded. The two observation points were treated as between-
subject measures because we could not certify that the same students sat in
the same seats for both discussions.
Dallimore et al. 15
Statistical Analysis. Univariate and multivariate analyses were conducted to test

for differences in student perceptions (based on survey data) and behavior
(based on observation data) between the two cold-call environments.
Results
Cold-Call Environment and Prior Student Attitudes and
Perceptions of Class Discussion
Based on the precourse survey data, there were no significant differences
between students in Lo CC and Hi CC sections at that time, as shown in
Table 1. Although students self-select into particular course sections based
on the course time and sequence, there does not appear to be a difference in
student characteristics a priori with respect to important features of class
discussion. Most notably, students in both environments had similar percep-
tions of their own participation in class discussion in terms of frequency,
satisfaction, liking, and comfort participating. Furthermore, there were no
differences between the two types of cold-call environments with respect to
expectations about participation in this course.
Effect of Cold-Calling on Percentage of Students

Voluntarily Responding to Questions
The primary evidence examining the effect of cold-calling on voluntary
responses to questions comes from the analysis of observation data collected
during two case discussions. The percentage of students present who
answered questions voluntarily was analyzed using a full factorial repeated-
measures ANOVA with two independent factors—cold-call environment (Hi
CC vs. Lo CC) and time of observation (Observation 1 vs. Observation 2).
The group means by cold-call environment and time of observation are
shown in Figure 2 along with the F-test results of the ANOVA. The analysis
of the percentage of students present who voluntarily answered questions
revealed three significant effects, as shown in Figure 2: a main effect due to
cold-call environment (Hi CC vs. Lo CC), a main effect due to time of obser-
vation (Observation 1 vs. Observation 2), and an interaction between cold-
call environment and observation. The main effect due to cold-call
environment shows that across both observation periods the mean percentage
of students who voluntarily answered questions was significantly higher in
the Hi CC sections (77%) than in the Lo CC sections (55%). Also, the per-
centage of students averaged across both cold-call environments who voluntarily
Table 1. Precourse Survey Mean Comparisons Between Hi CC and Lo CC

Sections for Key Measures of Participation Frequency and Comfort With Class
Discussion
Mean: Lo CC Mean: Hi CC Difference:

Sections (n = 241) Sections (n = 118) p Value
Liking of class 5.20 5.14 ns
discussion
Familiarity with class 5.78 5.87 ns
discussion
Typical participation 4.74 4.81 ns
frequency
Expected 5.07 5.11 ns
participation
frequency
Typical comfort 4.83 4.78 ns
participating
General evaluation of 4.96 5.00 ns
class participation
Satisfaction with class 4.78 4.87 ns
participation
Note: Hi CC = high cold-call; Lo CC = low cold-call.
answered questions rose from the first class discussion (62%) to the second
class session (71%). Finally, there was a significant interaction between
cold-call environment and observation such that there was a divergence
between the cold-call environments. Whereas the percentage of students who
voluntarily answered questions increased significantly in the Hi CC sections,
t(350) = −4.026, p = .000, the percentage of students who voluntarily
answered questions remained the same in the Lo CC sections from
Observation 1 to Observation 2, t(784) = 0.034, p = .973.
Furthermore, an indirect measure of the percentage of students participat-
ing in class discussion is obtained from the postcourse survey question which
asks students about their perception of other students’ participation. Students
in Hi CC sections rated the number of other students participating in this
course as compared with other courses significantly higher than students in
the Lo CC sections (Table 2). This confirms that students perceive the differ-
ences in percentage of students participating in class discussion that were
revealed by the observation data.
Dallimore et al. 17
Percent Students Voluntarily

Answered Ques!ons
1
0.86
Hi CC
(mean = .77)
0.8
0.68
0.6
Lo CC
0.55 0.55 (mean = .55)
0.4
0.2
0
Obs 1 (mean = .62) Obs 2 (mean = .71)
F p-value
Observation main effect 8.395 0.004
CC main effect 52.399 0.000
Observation x CC interaction 8.629 0.003
Figure 2. Percentage of students who voluntarily answered questions
Hypothesis 1a focuses on the relationship between cold-call environment and

the percentage of students volunteering during class discussion. The results pro-
vide clear support that cold-calling positively affects student volunteer behavior
as the mean percentage of students who voluntarily answered questions in Hi CC
sections is higher than in the Lo CC sections. Furthermore, Hypothesis 1b
addresses the impact of cold-calling on student volunteering behavior over time.
The results described above provide strong evidence for Hypothesis 1b. Most
notable, the mean percentage of students volunteering in the Hi CC sections
increases significantly from Observation 1 (68%) to Observation 2 (86%). In
contrast, the percentage of students volunteering in the Lo CC sections remains
stable at 55%. Thus, the analysis of the percentage of students volunteering dur-
ing class discussion supports both Hypotheses 1a and 1b.
Table 2. Postcourse Survey Mean Comparisons Between Hi CC and Lo CC

Sections for Key Measures of Participation Frequency and Comfort With Class
Discussion
Mean: Lo CC Mean: Hi CC Difference:

Sections (n = 241) Sections (n = 118) p Value
My actual participation 4.54 5.06 .003
frequency
Number of other students 4.37 5.04 .000
participating compared
with other courses
Comfort participating in 4.93 5.06 ns
this course
Note: Hi CC = high cold-call; Lo CC = low cold-call.
Effect of Cold-Calling on Frequency

of Student Class Participation
Whereas previously the analysis has focused on the percentage of students
participating in class discussions, this section focuses on the number of ques-
tions being answered. Frequency of class participation was measured in two
ways. First, using the observation data, the number of questions answered
per student was calculated. Second, using the survey data, student self-
reported actual frequency of class participation was recorded.
Observations of class room behavior. Counts of the number of questions
answered per student were analyzed using a full factorial univariate ANOVA
with two independent factors—cold-call environment and observation. The
analysis of number of volunteer questions answered per student is shown in
Figure 3. The results reveal two significant main effects due to cold-call
environment and to observation and a significant interaction effect. The
main effect due to cold-call environment shows that the number of volunteer
questions answered per student is significantly higher in Hi CC sections
(2.25) than in Lo CC sections (1.60). Furthermore, the main effect due to
observation period shows that students answer significantly more volunteer
questions in the second observation period (2.27) than in the first (1.59).
Finally, the interaction indicates that in the Hi CC sections there is a signifi-
cant increase in the number of volunteer questions answered per student
over time, t(350) = −5.317, p = .000, whereas the number of volunteer ques-
tions answered per student in the Lo CC sections does not significantly
change, t(784) = −1.185, p = .237.
Dallimore et al. 19
Number of Volunteer
Ques!ons Per Student
2.84
3 Hi CC
2.5 (mean = 2.25)
2 Lo CC
1.66
1.5 (mean = 1.60)
1.69
1.51
1
0.5
0
Obs 1 (mean = 1.59) Obs 2 (mean = 2.27)
F p-value
Figure 3. Number of volunteer questions per student
The results from the analysis of the number of volunteer questions per
student strongly support both Hypotheses 2a and 2b. Hypothesis 2a relates
cold-calling to voluntary student class participation. The mean number of
voluntary questions answered per student was significantly higher for stu-
dents in Hi CC sections (2.25) compared with Lo CC sections (1.60).
Hypothesis 2b contends that the frequency of voluntary class participation
per student increases over time in Hi CC sections but does not change in Lo
CC sections. This is precisely the pattern of the significant interaction found
between cold-call environment and observation period: The number of vol-
unteer questions answered per student rose significantly from 1.66 to 2.84 in
Hi CC sections, but the change for Lo CC sections was not significant,
increasing slightly from 1.51 to 1.69.
To understand students’ overall frequency of participation, we need to
know not only how many times they participate voluntarily but also how
often they were cold-called. Thus, the two variables, number of voluntary
questions answered per student and the number of cold-called questions
Sum of Voluntary and Cold-Call

(Overall) Ques!ons Per Student
4
3.5 3.34 Hi CC
(mean = 2.81)
3
2.5 2.27
2
Lo CC
1.5 1.77
1.61 mean = (1.69)
1
0.5
0
Obs 1 (mean = 1.94) Obs 2 (mean = 2.56)
F p-value

Figure 4. Sum of voluntary and cold-call (overall) questions per student
answered per student, together provide a comprehensive description of the

frequency of a student’s class participation. Analysis of the sum of these two
variables revealed two significant main effects, due to cold-call environment
and to observation period, as well as a significant interaction effect, as
shown in Figure 4. The main effect due to cold-call environment shows that
there were significantly more questions answered per student, on average, in
Hi CC sections (2.81) than in Lo CC sections (1.69). Furthermore, the mean
overall number of questions answered per student was significantly higher
during the second observation period (2.56) than the first (1.94). Finally, the
cold-call environment interacted with the time of observation. In the Hi CC
sections, there was a significant increase from the first to the second obser-
vation (2.27 vs. 3.34) whereas in the Lo CC sections, the two observations
remained stable (1.61 vs. 1.77).
This combined variable of overall participation frequency exhibits the
same pattern as was found for the number of voluntary questions answered
per student. This is the case because instructors ask many more voluntary
Dallimore et al. 21
questions than cold-called questions in both environments. Thus, even

though a significantly higher mean number of cold-calls per student were
observed in Hi CC sections than in Lo CC sections (0.56 vs. 0.09), these
numbers are small relative to the respective number of volunteer questions
per student (2.25 vs. 1.60). Given these results, we would expect that stu-
dent perception of their class participation would reflect the pattern found
for the combined voluntary and cold-called questions answered during
class discussion.
Student self-reported participation frequency. Turning to evidence from the
survey data, no differences were found between Hi CC and Lo CC sections
with respect to self-report of one’s own typical participation frequency from
the precourse survey (Table 1). However, students’ self-reported ratings of
their perception of their actual participation frequency in the course from the
postcourse survey revealed a statistically significant difference between the
Hi CC and Lo CC sections (Table 2).
A repeated-measures ANOVA was conducted for students’ self-reported
perception of their participation frequency, with the precourse and post-
course self-reported survey variables as the repeated measure and the cold-
call environment as the between-subject variable. Figure 5 contains the
analysis for participation frequency, comparing the students’ perceived
typical participation frequency from the precourse survey with their self-
reported actual participation frequency in this course from the postcourse
survey. The results indicate a significant cold-call environment main effect
and a significant interaction between cold-call environment and survey
period. Self-reported participation frequency was significantly higher in Hi
CC sections (4.94) than in Lo CC sections (4.64). As shown in Figure 5,
there was a divergence of self-reported participation frequency at the end of
the course such that students in the Hi CC sections marginally increased
their assessment of their participation frequency from 4.81 to 5.06, t(117) =
−1.823, p = .071, whereas students in the Lo CC sections significantly
decreased their participation frequency ratings relative to their assessment
at the beginning of the course from 4.73 to 4.54, t(238) = 2.201, p = .029.13
Hypothesis 2c relates cold-calling environment to overall frequency of
student participation. The results provide clear support that cold-calling posi-
tively affects overall student participation frequency. First, students in the Hi
CC sections significantly increased the number of overall questions answered
from the first observation to the second, whereas the number of overall ques-
tions answered by students in the Lo CC sections remained stable across
time. Second, the magnitude of the difference in participation frequency was
sufficient for students to perceive the difference. Student self-reported
Self-Reported
Par!cipa!on Frequency
5.1 5.06 Hi CC
(mean = 4.94)
5
4.9 4.81
4.8
4.7
4.73
4.6
Lo CC
4.5
4.54 (mean = 4.64)
4.4
4.3
4.2
Pre-Course (mean = 4.77) Post-Course (mean = 4.80)
F p-value
Survey main effect 0.152 0.697

Survey x CC interaction 7.954 0.005
Figure 5. Self-reported participation frequency
participation frequency diverged between the two cold-call environments

such that students in the Hi CC sections reported marginally higher participa-
tion frequency at the end of the course compared with the beginning of the
course, whereas those in Lo CC sections reported significantly less participa-
tion frequency at the end of the course compared with the beginning of the
course. Thus, the evidence strongly supports the contention that cold-calling
is positively related to increases in overall participation frequency in class
discussion. Furthermore, data in this study clearly indicate that the high cold-
call environment is positively and significantly related to overall frequency
of student participation in class discussion and that the frequency of class
participation increases in Hi CC sections over the course of the term.
Effect of Cold-Call Environment on Comfort With Class

Discussion
Given that cold-calling affects observed student participation behavior,
including the percentage of students participating and frequency of student
participation, the question of how cold-calling affects student comfort is of
Dallimore et al. 23
Self-Reported Comfort
with Class Discussion
5.1 5.06 Hi CC
5.05 (mean = 4.92)
5
4.95
Lo CC
4.9 4.93 (mean = 4.89)
4.84
4.85
4.8
4.75 4.78
4.7
4.65
4.6
F p-value
Figure 6. Self-reported comfort with class discussion
interest. As discussed above, there were no differences found between Hi

CC and Lo CC sections when assessing self-reported comfort with class
discussion from the precourse survey. Given that differences observed due
to cold-call environment with respect to question asking and answering
behavior are notable enough that students perceive and report the differ-
ences, students may also experience different levels of comfort due to cold-
call environment.
To assess differences, we systematically analyzed the independent effects
of cold-call environment and survey time period on self-reported measures of
comfort with class discussion. A repeated-measures ANOVA was conducted
for comfort, with the precourse and postcourse self-reported survey variables
as the repeated measure and the cold-call environment as the between-subject
variable; the results are shown in Figure 6. Only the main effect due to survey
period was statistically significant. Students overall reported an increase in
comfort with class discussion. Although Figure 6 shows that comfort ratings
rose more for students in Hi CC sections than for those in Lo CC sections, the
interaction was not statistically significant.
Hypothesis 3 deals with the effect of cold-calling on student comfort with

class discussion. The results provide no evidence that students’ self-reported
comfort with class discussion is significantly affected by the cold-call
environment.14
Sensitivity Analysis
The results examining the effect of cold-calling on student class participa-
tion behavior and their perceptions of class discussion are based on one
approach to categorizing sections into Hi CC and Lo CC groups. We won-
dered how sensitive the results reported above are to this grouping decision
rule. To maximize the difference between Hi CC and Lo CC sections, we
split the sample into thirds with respect to the mean percentage of students
cold-called during the first observation. We designated the top third of the
sections as Hi CC sections and the bottom third, as Lo CC sections. The
middle third of the sections were omitted from this analysis. This grouping
rule resulted in five sections with 196 students rated as Hi CC sections and
five sections with 210 students, as Lo CC sections. The mean percentage of
students cold-called in the Hi CC sections was .61 with a range from .33 to
.84 and in the Lo CC sections, .01 with a range from .00 to .02.
The analyses based on this maximally differentiated sample (with respect
to cold-call behavior) revealed very similar results for the percentage of stu-
dents answering voluntary questions and the number of voluntary questions
answered per student (see Figures 2 and 3), observed frequency of class par-
ticipation (see Figure 4), and student self-reported participation frequency
(see Figure 5). There were, however, different results with respect to student
comfort.
As shown in Figure 7, the repeated-measures ANOVA showed that nei-
ther the survey period main effect nor the cold-call environment main
effect was statistically significant. However, the two-way interaction was
statistically significant, showing a divergence between the two cold-call
environments over time with respect to self-reported comfort with class
discussion. Further analysis of the interaction found that students in Hi CC
sections increased their comfort ratings marginally from the precourse to
the postcourse survey, t(117) = −1.885, p = .062, whereas student ratings
of comfort with class discussion in the Lo CC sections did not change,
t(113) = 1.243, p = .216. Additionally, although the traditional t test shown
in Table 2 measuring mean comparisons between Hi CC and Lo CC sec-
tions for the postcourse survey responses did not yield a significant differ-
ence between cold-call environments, the significant interaction discussed
Dallimore et al. 25
Sensi!vity Analysis:
Self-Reported Comfort
with Class Discussion
5.1 5.06 Hi CC
5.05 (mean= 4.92)
5 4.95
4.95
4.9
4.85
Lo CC
4.8 (mean = 4.86)
4.75
4.78 4.77
4.7
4.65
4.6
F p-value
Figure 7. Sensitivity analysis: Self-reported comfort with class discussion
above enables a post hoc comparison between cold-call environments in

the postcourse survey to be conducted. A 95% Scheffé confidence inter-
val revealed a significant difference in the mean comfort rating between
Hi CC sections and Lo CC sections (.003 < µ1 − µ2 < .571; Dixon &
Massey, 1969).
The different results between the full sample and the differentiated sample
that maximizes differences between Hi CC and Lo CC sections seems to rest
on the relationship between precourse and postcourse student self-reported
comfort ratings for the Lo CC sections. In the full sample, the two-way inter-
action was not significant, but in the differentiated sample, it was. In both
cases, student comfort ratings increased marginally for Hi CC sections. Ad
hoc analysis of the Lo CC sections in the full sample revealed that there was
no significant increase in comfort ratings for the Lo CC sections alone, t(239)

= −.947, p = .345, which is similar to results reported above for Lo CC sec-
tions in the differentiated sample. Thus, in both samples, student comfort with
class discussion ratings were stable from the precourse survey to the post-
course survey for the Lo CC sections but increased marginally for the Hi CC
sections.
This sensitivity analysis confirms earlier results concerning Hypotheses
1a, 1b, 2a, 2b, and 2c. Furthermore, it provides evidence that the cold-call
environment affects students’ self-reported comfort with class discussion
(Hypothesis 3). The significant interaction between observation period and
cold-call environment for the differentiated sample indicates that in the Hi
CC sections, student ratings of comfort increase, whereas in the Lo CC sec-
tions, they do not change.
Discussion and Implications

Cold-calling has a significant effect on the number of students who volun-
tarily answer questions. As we know from the precourse survey, students
in both high and low cold-calling sections enter the course with the same
typical participation frequency in prior courses, and they expect to par-
ticipate the same amount in this course. By the time the first observation
is conducted in Weeks 6 to 7, the instructor’s questioning pattern is likely
well established. At that point, more students in the high cold-calling sec-
tions are already voluntarily answering questions, and by the second
observation (Weeks 9-10) the percentage of students who voluntarily
answer questions in the high cold-calling sections continues to increase
and diverge from that in the low cold-calling sections. Furthermore, stu-
dents in the high cold-calling sections are simply answering more ques-
tions, overall, than students in the low cold-calling sections. Also, the
percentage of students who participate in class discussions increases quite
dramatically from just more than half in low cold-calling sections to just
more than 90% in high cold-calling sections.
These findings suggest greater engagement of students in the high cold-
calling sections. These differences are not only visible to trained observers
but are also perceived by students themselves. Students in the high cold-
calling sections reported an increased participation rate for themselves and
they reported that more than the typical number of other students participated
in the course.
Finally, cold-calling does not have the dire impact on student comfort
that the instructors cited earlier had feared. In fact, what we find is that
Dallimore et al. 27
students in high cold-calling sections are more comfortable participating in

this course than they were participating in previous courses, whereas the
comfort of students in the low cold-calling sections does not change.
Increased comfort participating in class discussions in high cold-calling
environments may seem counterintuitive, as it seems unlikely that cold-
calling per se makes students more comfortable. This study shows that
high cold-calling is associated with increased frequency of participation,
especially voluntary participation. Both participation frequency and prep-
aration have previously been shown to be associated with increased com-
fort participating (Dallimore et al., 2010). Thus, a more plausible
explanation is that cold-calling encourages students to prepare more and
to participate more frequently; the more they prepare, and the more
frequently they participate, the more comfortable they become when
participating.15
Implications for Teaching

The clear implication for teaching is that cold-calling is a way to engage
more students more actively in class discussions and that this can be done
without necessarily making them less comfortable participating. These find-
ings are especially important in light of the learning model presented in
Dallimore et al. (2010). That model shows positive relationships between
preparation, participation frequency, comfort participating in class discus-
sions, and student learning. In this study, we find that cold-calling increases
students’ participation frequency and comfort participating, and some evi-
dence that it also increases preparation for class. Thus, cold-calling
increases those factors which have previously been shown to be associated
directly with increased learning.
We do not suggest, however, that any and all cold-calling by instructors
would produce the same results as those shown in this study. As with any
other instructional technique, instructors must be thoughtful about how
they use cold-calling. Dallimore et al. (2005) reports instructional strate-
gies used by experienced instructors to “warm up” cold-calling so that it is
effective and nonpunitive. These strategies include establishing the expec-
tation of participation (e.g., stating expectations in the syllabus, telling
students you will cold-call, cold-calling in first class), providing opportu-
nities to reflect and respond (e.g., give them time to think and prepare, let
small groups of students discuss the answer first), and creating a support-
ive learning environment (e.g., encourage with body language, reinforce
any reasonable attempt).
Furthermore, given that class discussion skills develop over time, stu-
dents must first simply learn to speak before they can develop the ability
to contribute in a more significant and thoughtful way. Thus, instructors
might use a hierarchy such as Bloom’s taxonomy (Anderson & Krathwohl,
2001; Bloom, 1956) to guide the development of student class discussion
skills and enhance their confidence participating. They might cold-call
early in the course using straightforward knowledge or recall questions to
make sure each student’s voice is heard and that each student has a
successful participation experience. Building on this the questions can
become more challenging—eventually progressing to analysis and evaluation—
while creating a series of successes through which a student can develop
the skills to participate more substantively and the confidence to par-
ticipate voluntarily. As more students are engaged more actively in
class discussion, the discussion benefits from more voices being
heard and from the greater diversity of thought and of experience that
they represent.
Clearly, this study has implications for discussion facilitation more gen-
erally (e.g., encouraging instructors not simply to rely on volunteers and to
understand the available range of nonvoluntary participation strategies).
Additionally, there are clear implications for faculty development and
teacher training. For example, findings provide support not only for the
development and use of classroom discussion pedagogies that engage more
reluctant participants in discussion activities but also support the value of
helping faculty learn how to effectively use these strategies. Furthermore,
relative to student-skill development and program design, we suggest that a
systematic use of cold-calling could be an informal but systematic way to
help communication-apprehensive students feel more comfortable speak-
ing in front of their instructors and peers.
Limitations and Future Research

The results of this study should be interpreted in light of its limitations.
First, student data were gathered only for one course with multiple sections
and instructors. Thus, our ability to generalize to other courses is limited.
Second, we do not know the exact nature of the questions asked by instruc-
tors in the high cold-calling sections that enabled them to increase the
voluntary response rate and overall participation rate without reducing
student comfort. Although Dallimore et al. (2005) contains some guidance
along this line, more research is needed. One area for future inquiry would
Dallimore et al. 29
be an exploration of the relationship among specific questioning-and-

answering patterns. For example, does cold-calling use at the higher levels
of Bloom’s taxonomy increase voluntary participation while use at its
lower levels—where there are arguably a more narrow range of acceptable
responses (i.e., right or wrong answers)—inhibit voluntary participation?
Another example would be to explore the relationship between cold-calling
at various levels of Bloom’s taxonomy and answering patterns that might
be characterized as chains of student-to-student contributions.
In addition, since we did not differentiate among students in this study,
we do not know whether our findings hold in the same way for male as well
as female students or for students from different cultures. Again, additional
research could illuminate these issues by testing for gender effects on cold-
calling versus voluntary-participation patterns. One such area for future
inquiry would be to assess cold-calling versus voluntary participation based
on the gender of both the instructor and the students, especially since dif-
ferences in participation patterns for gender have already been documented
in in-class and online environments (Caspi et al., 2006; Howard et al.,
2006). Such questions might include the following: Are instructors more
likely to cold-call on male students? If so, do instructors also tend to direct
more difficult questions toward the male students? What about soliciting
voluntary participation across Bloom’s taxonomy based on gender? If dif-
ferences exist, does the gender of the instructor influence these outcomes?
Furthermore, although our data do not explain why the percentage of
students voluntarily answering questions is so much higher in the high
cold-calling sections, our informal discussions with students suggest three
possibilities, which can be characterized as “priming the pump,” “comfort
in numbers,” and “defensive volunteering.” First, once a student has par-
ticipated a few times, it becomes easier to participate, including volunteer-
ing; this explanation is akin to priming a pump. Second, if only a few
students volunteer, additional students might feel uncomfortable joining
this visible minority. However, when the percentage of students volun-
tarily answering questions approaches 90%, then volunteering is the norm,
and students may find that there is comfort in numbers. Finally, when
instructors cold-call, students who prefer not to be cold-called may volun-
teer more frequently. They use a defensive strategy for participation rea-
soning that if they volunteer often, their instructor is less likely to cold-call
them. We suspect that each of these three explanations is true to some
degree for at least some students. Further research is needed to determine
the strength and validity of these possible explanations.
Furthermore, our study showed the longitudinal development of greater

participation across the course. This raises interesting questions about the
effects beyond the boundary of a single course, that is, the impacts on
program design. If this pedagogy were used in the introductory courses of
(or better yet consistently throughout) an academic course of study, stu-
dents could develop confidence in their ability to express themselves orally
in a more low-stakes and ongoing way before being required to complete a
more formalized speaking assignment or even more explicitly communication-
focused course (e.g., public speaking, business communication).
Finally, with an increasing emphasis on the development and delivery of
online courses, future research might examine the form cold-calling might
take in an online learning environment. For example, such research might
examine strategies for engaging less willing participants (e.g., posing a
question to a particular student in a chat session rather than to the entire
class) as well as an examination of how cold-calling use, through both syn-
chronous and asynchronous discussion pedagogies, might affect variables
such as student preparation, participation behavior, and learning.
Summary
This study examined the effect of cold-calling on students’ participation in
class discussions, especially voluntary participation, and on their comfort
participating. Pre- and postcourse surveys concerning student perceptions
about and behaviors during class discussion in a required business course
were gathered from sophomore students at a large, private research univer-
sity. In addition, trained observers gathered data on instructor question-asking
behavior during two case discussions.
The primary results indicate that in high cold-calling environments,
more students participate voluntarily, the number of students participating
voluntarily increases over time, and more questions are asked and
answered. Furthermore, more students participate overall in class discus-
sions in high cold-calling environments. Finally, comfort participating
increases in high cold-calling environments; it does not change in low
cold-calling environments.
That cold-calling has such a significant, positive effect on participation
is an important and valuable extension of literature related to discussion
teaching in general and cold-calling in particular. Furthermore, the fact that
cold-calling can be done fairly extensively without making students uncom-
fortable provides support for using this instructional strategy to engage
more students to participate more actively in class discussions.
Dallimore et al. 31
Appendix
Survey Variables Survey Questions
Precourse survey
Liking of class discussion How much do you like class discussion? 1 = not at all
and 7 = very much
Familiarity with class discussion How familiar are you with class discussion? 1 = not
familiar and 7 = very familiar
Typical participation frequency In most courses, I participate: 1 = not at all and 7 = very
frequently
Expected participation In this course, I expect to participate: 1 = not at all and
frequency 7 = very frequently
Typical comfort participating In general, when I participate in class discussions, I feel:
1 = uncomfortable and 7 = comfortable
General evaluation of class In general, the evaluation of my class participation has
participation been: 1 = low and 7 = high
Satisfaction with class In general, my satisfaction with my class participation is:
participation 1 = not satisfied and 7 = very satisfied
Postcourse survey
My actual participation In this course, I actually participated: 1 = not at all and 7
frequency = very frequently
Number of other students Compared with other courses, the number of students
participating compared with who participated in class discussion was: 1 = much
other courses lower and 7 = much higher
Comfort participating in this In this course, when I participated in class discussions I
course felt: 1 = less comfortable and 7 = more comfortable
Observation Variables Variable Definition
Percentage students participated Total number of students answering volunteer or cold-

in class discussion called questions/total students present
Percentage students cold-called Total number of students answering cold-called
questions/total students present
Percentage students voluntarily Total number of students answering volunteer
answering questions questions/total students present
Total number of questions per Total number of questions asked/total students present
student
Number of volunteer questions Total number of volunteer questions asked/total
per student students present
Number of cold-called questions Total number of cold-called questions asked/total
per student students present
Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,
authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or pub-
lication of this article.
Notes
1. There is extensive literature on learning styles and how these learning styles
affect discussions and conversations. See, for example, Claxton and Murrell
(1987), Rayner and Cools (2011), Sadler-Smith and Evans (2006), and Sims and
Sims (1995).
2. We have seen rapid movement toward technology in teaching and as a tool for
student learning. It has been used to engage students in discussion for varied
reasons (e.g., it allows for more participation without the constraints of class
time and allows students more control over both the content and context of their
participation). The impact of technology on class interaction is investigated in
the field of management by Arbaugh (2000) who examines participation in an
asynchronous Internet-based MBA course and finds no significant differences
in learning or interaction quality from the conventional classroom. However, he
finds significantly more participation overall, a finding consistent with Bump’s
earlier conclusions.
3. The four teachers in the study are all authors on the Souza et al. (2010) article;
they collaborated and agreed to follow consistent instructional practices with the
exception of the use of cold-calling. The authors note that “the fact that each of
the four instructors work diligently to create and maintain a supportive commu-
nication climate and have been recognized for effective teaching likely impacted
the results” (Souza et al., 2010, p. 245). Furthermore, there were significant dif-
ferences in the precourse variables between the students in courses with cold-
calling and the students in courses without cold-calling. This suggests that these
groups of students were quite different before the experiment, which may also
affect the results.
4. This discrepancy may stem from instructors’ extrapolating their past experience
with a particular student at a point in time, rather than understanding the impact
of cold-calling on students’ comfort in general, and comprehending the effect of
this pedagogy on student comfort over time.
5. This is consistent with Bandura’s (1997) theory of self-efficacy.
6. Although we attempted to control for as many variables as possible to eliminate
alternative explanations of our findings, this research was conducted in actual
Dallimore et al. 33
classes and was not a controlled experiment. Thus, other contextual factors (e.g.,
setup of the classroom, time of day of class) could possibly influence the results.
7. Prior to conducting this research study, two of the authors observed many discus-
sion sections in this course to develop instruments and coding schemes for the
study. They observed instructors asking questions related to all levels of Bloom’s
taxonomy. For example, knowledge/recall (How do fixed costs behave when
production increases?); comprehension (Why do fixed costs per unit decrease
when production increases?); application (If you double production, what hap-
pens to variable costs?); analysis/synthesis (What is really going on here? What
might be the reason for an unfavorable price variance and a favorable usage
variance? What can you conclude from these data? How did you come to that
conclusion?); and evaluation (What would be the arguments for using the [. . .]
method? Do you agree with Susan? Why? Did you see any problems with this
approach? Do you think this is realistic? What are the pros and cons? How would
you summarize what we have said so far?).
8. The remaining students may have been present for one survey and not the other,
or they may have forgotten, incorrectly remembered, or chosen not to reveal the
last four digits of their student identification number. In these instances, data from
the precourse and the postcourse surveys could not be matched, and they were
eliminated from the analysis sample. For the variables examined in this study
(Table 1) we conducted mean comparison tests between matched and unmatched
samples. No significant differences between the two groups were found.
9. Teaching effectiveness ratings are independent of this project. They were gath-
ered from the university’s teaching evaluation program.
10. “Treats student with respect” ratings are independent of this project. They were
gathered from the university’s teaching evaluation program.
11. Dates that surveys would be administered and observations conducted (as shown
in Figure 1) were discussed with the instructors to minimize disruption. Since
observations were planned when instructors had scheduled case discussions, vir-
tually no changes occurred in the observation dates; however, survey dates were
sometimes adjusted by one class session to ensure sufficient time was available
to administer a survey.
12. This particular postcourse survey question was only asked in the second year of
the 2-year study period.
13. It may be that the analyses reported for participation frequency could be
affected by the majority of students who really like class discussion. Out of
concern that we might be overlooking the effect of cold-calling on students
who do not like class discussion, we conducted the analyses on participation
frequency for just those students who reported that they did NOT like class
discussion on the precourse survey. Although this substantially reduced the
sample size, it produced similar results. As with the larger sample, students in
the Hi CC sections rated their participation frequency significantly higher than

students in the Lo CC sections. Although the interaction between CC section
and survey period was not significant, the pattern of results was similar, in
that students in the Hi CC sections increased their self-reported participation
frequency ratings over time, whereas students in the Lo CC sections reported
stable ratings of participation frequency.
14. Students who are most uncomfortable with class discussion may simply choose
to not attend those sessions. Z tests comparing percentage of students attending
class by cold-call environment revealed no difference in percentage of students
attending class by cold-call environment for the first class discussion (Z = .15,
p = .881) or the second class discussion (Z = .12, p = .904). Thus, class atten-
dance is not related to cold-call environment, that is, cold-calling does not
dissuade students from attending class.
15. We had limited data on student self-reported preparation for class on which we
could conduct ad hoc auxiliary analysis because this particular postcourse sur-
vey question was only asked in the second year of the 2-year study period. The
results showed that students in Hi CC sections rated the impact of cold-calling
on their preparation for class (4.71) marginally higher than students in Lo CC
sections (4.29), t(108) = −.77, p = .080.
References
10 Tips for Public Speaking. (n.d.). Retrieved from http://www.toastmasters.org/
MainMenuCategories/FreeResources/NeedHelpGivingaSpeech/TipsTechniques/
10TipsforPublicSpeaking.aspx
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching and
assessing: A revision of Bloom’s taxonomy of educational objectives. New York,
NY: Longmans.
Arbaugh, J. B. (2000). The virtual classroom versus physical classroom: An
exploratory study of class discussion patterns and student learning in an asyn-
chronous internet-based MBA course. Journal of Management Education, 24,
213-233.
Arbaugh, J. B., & Benbunan-Fich, R. (2006). An investigation of epistemological
and social dimensions of teaching in online learning environments. Academy of
Management Learning & Education, 5, 435-447.
Arter, J., & McTighe, J. (2001). Scoring rubrics in the classroom. Thousand Oaks,
CA: Corwin.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Jossey-Bass.
Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of
educational goals. Book 1, Cognitive domain. New York, NY: Longman, Green.
Dallimore et al. 35
Bonwell, C., & Eisen, J. (1991). Active learning: Creating excitement in the classroom
(ASHE-ERIC Higher Education Report No. 1). Washington, DC: Association for
the Study of Higher Education.
Brower, H. H. (2003). On emulating classroom discussion in a distance-delivered
OMHR course: Creating an on-line learning community. Academy of Manage-
ment Learning & Education, 2, 22-36.
Bruns, W. J., Jr. (2006). Why I use the case method to teach accounting (Case method
Ref. 9-193-177). Boston, MA: Harvard Business School Press.
Bump, J. (1990). Radical changes in class discussion using networked computers.
Computers and the Humanities, 21, 49-65.
Carnaghan, C., & Webb, A. (2007). Investigating the effects of group response sys-
tems on student satisfaction, learning, and engagement in accounting education.
Issues in Accounting Education, 22, 391-409.
Caspi, A., Chajut, E. & Saporta, K. (2008). Participation in class and in online discus-
sions: Gender differences. Computers & Education, 50, 718-724.
Christensen, C. M., & Carlile, P. R. (2009). Course research: Using the case method
to build and teach managment theory. Academy of Management Learning &
Education, 8, 240-251.
Christensen, C. R. (1991). Every student teaches and every teacher learns: The
reciprocal gift of discussion teaching. In C. R. Christensen, D. A. Garvin, &
A. Sweet (Eds.), Education for judgment: The artistry of discussion leadership
(pp. 99-122). Boston, MA: Harvard Business School Press.
Christensen, C. R., & Hansen, A. J. (1987). Teaching and the case method: Text,
cases, and readings. Boston, MA: Harvard Business School Press.
Claxton, C. S., & Murrell, P. H. (1987). Learning styles: Implications for improv-
ing educational practices (ASHE-ERIC Higher Education Report No. 4). College
Station, TX: Association for the Study of Higher Education.
Cunningham, B. M. (2008). Using action research to improve learning and the class-
room learning environment. Issues in Accounting Education, 23, 1-30.
Dallimore, E. J., Hertenstein, J. H., & Platt, M. B. (2004). Classroom participation and
discussion effectiveness: Student generated strategies. Communication Education,
53, 103-115.
Dallimore, E. J., Hertenstein, J. H., & Platt, M. B. (2005). Faculty-generated strategies
for “cold calling” use: A comparative analysis with student recommendations. Jour-
nal on Excellence in College Teaching, 16, 23-62.
Dallimore, E. J., Hertenstein, J. H., & Platt, M. B. (2006). Nonvoluntary class partici-
pation in graduate discussion courses: Effects of grading and cold calling. Journal
of Management Education, 30, 354-377.
Dallimore, E. J., Hertenstein, J. H., & Platt, M. B. (2008). Using discussion peda-
gogy to enhance oral and written communication skills. College Teaching, 56,
163-172.
Dallimore, E. J., Hertenstein, J. H., & Platt, M. B. (2010). Class participation in
accounting courses: Factors that affect student comfort and learning. Issues in
Accounting Education, 25, 613-629.
Davis, B. G. (2009). Tools for teaching (2nd ed.). San Francisco, CA: Jossey-Bass.
Dixon, W. J., & Massey, J. F. (1969). Introduction to statistical analysis (3rd ed.).
New York, NY: McGraw-Hill.
Ellis, R. A., Goodyear, P., Prosser, M., & O’Hara, A. (2006). How and what university
students learn through online and face-to-face discussion: Conceptions, intentions
and approaches. Journal of Computer Assisted Learning, 23, 244-256.
Ewens, W. (2000). Teaching using discussion. In R. Neff & M. Weimer (Eds.), Class-
room communication: Collected readings for effective discussion and questioning
(pp. 21-26). Madison, WI: Atwood.
Fishman, S. M. (1997). Student writing in philosophy: A sketch of five techniques.
New Directions for Teaching and Learning, 69, 53-66.
Gilmore, T. N., & Schall, E. (1996). Staying alive to learning: Integrating enactments
with case teaching to develop leaders. Journal of Policy Analysis and Manage-
ment, 15, 444-457.
Greenhalgh, A. M. (2007). Case method teaching as science and art: A metaphoric
approach and curricular application. Journal of Management Education, 31, 181-194.
The Habit of Courage. (n.d.). Toastmasters International. Retrieved from http://www
.toastmasters.org/MainMenuCategories/FreeResources/NeedHelpGivingaS-
peech/FearFactor/TheHabitofCourage.aspx
Hertenstein, J. H. (1991). Patterns of participation. In C. R. Christensen, D. A. Garvin,
& A. Sweet (Eds.), Education for judgment: The artistry of discussion leadership
(pp. 175-191). Boston, MA: Harvard Business School Press.
Howard, J., Zoeller, A. & Pratt, Y. (2006). Students’ race and participation in sociol-
ogy classroom discussion: A preliminary investigation. Journal of Scholarship of
Teaching and Learning, 5(2), 14-38.
Liang, N., & Wang, J. (2004). Implicit mental models in teaching cases: An empirical
study of popular MBA cases in the United States and China. Academy of Manage-
ment Learning & Education, 3, 397-413.
Nyquist, J. D., & Wulff, D. H. (1990). Selected active learning strategies. In J. A. Daly,
G. W. Friedrich, & A. L. Vangelisti (Eds.), Teaching communication: Theory,
research and methods (pp. 337-362). Hillsdale, NJ: Lawrence Erlbaum.
Rayner, S., & Cools, E. (Eds.). (2011). Style difference in cognition, learning, and
management: Theory, research, and practice. New York, NY: Routledge.
Dallimore et al. 37
Rollag, K. (2010). Teaching business cases online through discussion boards: Strate-
gies and best practices. Journal of Management Education, 34, 499-526.
Rosmarin, A. (1987). The art of leading a discussion. In C. R. Christensen & A. J. Hansen
(Eds.), Teaching and the case method: Text, cases, and readings (pp. 235-240).
Boston, MA: Harvard Business School Publishing.
Sadler-Smith, E., & Evans, C. (Eds.). (2006). Learning styles in education and
training. Bradford, England: Emerald Group.
Schmidt-Wilk, J. (2010). Signature pedagogy: A framework for thinking about
management education. Journal of Management Education, 34, 491-494.
Scollon, S., & Bau, K. (1981, April). A model for making higher education more
culturally sensitive. Paper presented at the Conference of the National Associa-
tion for Asian and Pacific American Education, Honolulu, HI.
Sims, R. R., & Sims, S. J. (Eds.). (1995). The importance of learning styles: Under-
standing the implications for learning, course design, and education. Westport,
CT: Greenwood Press.
Smith, L. J., & Smith, D. L. (1994). The discussion process: A simulation. Journal of
Reading, 37, 582-584.
Souza, T. J., Dallimore, E. J., Aoki, E., & Pilling, B. C. (2010). Communication
climate, comfort and cold-calling: An analysis of discussion-based courses at
multiple universities. In L. B. Nelson & J. E. Miller (Eds.), To improve the acad-
emy: Resources for faculty, instructional, and organizational development (Vol.
28, 227-249). San Francisco, CA: Jossey-Bass.
Sprague, J., & Stuart, D. (2003). The speaker’s handbook (6th ed.). Belmont, CA:
Wadswoorth/Thomson Learning.
Stevens, D. D., & Levi, A. J. (2005). Introduction to rubrics. Sterling, VA: Stylus.
Weaver, R. R., & Qi, J. (2005). Classroom organization and participation: College
students’ perceptions. Journal of Higher Education, 76, 570-600.

Cold Calling

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cold Calling

Uploaded by

Copyright:

Available Formats

Journal of Management Education

Impact of Cold-Calling XX(X) 1–37

Elise J. Dallimore1, Julie H. Hertenstein1, and

For more than 20 years, educational philosophers and practitioners have

comfortable participating in discussions, whereas in classes with low cold-

discussions are preferred to lectures when training future business leaders

Students also tend to share the language system of their generation, a

Although some students may have developed sufficient class discussion

Cold-Calling: A Technique to Increase Class Participation

Furthermore, earlier studies have no measure of the number of students

Hypothesis 1a: Cold-calling positively affects the percentage of stu-

Another characteristic of student participation in class discussions is how often

Hypothesis 2b: The frequency that students participate voluntarily

A student’s overall frequency of participation includes the number of

Hypothesis 2c: Cold-calling positively affects the overall (voluntary

Despite instructor concerns that cold-calling will make students uncom-

Hypothesis 3: Student comfort participating in class discussion

Dependent Variables: Survey Data. As discussed above, two surveys were

mean percentage of students cold-called was calculated for each section; it

Statistical Analysis. Univariate and multivariate analyses were conducted to test

Effect of Cold-Calling on Percentage of Students

Table 1. Precourse Survey Mean Comparisons Between Hi CC and Lo CC

Mean: Lo CC Mean: Hi CC Difference:

Percent Students Voluntarily

Observation main effect 8.395 0.004

CC main effect 52.399 0.000

Observation x CC interaction 8.629 0.003

Figure 2. Percentage of students who voluntarily answered questions

Hypothesis 1a focuses on the relationship between cold-call environment and

Table 2. Postcourse Survey Mean Comparisons Between Hi CC and Lo CC

Mean: Lo CC Mean: Hi CC Difference:

Effect of Cold-Calling on Frequency

Observation main effect 25.461 0.000

CC main effect 23.292 0.000

Observation x CC interaction 13.868 0.000

Figure 3. Number of volunteer questions per student

Sum of Voluntary and Cold-Call

Observation main effect 20.713 0.000

Figure 4. Sum of voluntary and cold-call (overall) questions per student

answered per student, together provide a comprehensive description of the

questions than cold-called questions in both environments. Thus, even

Survey main effect 0.152 0.697

Figure 5. Self-reported participation frequency

participation frequency diverged between the two cold-call environments

Effect of Cold-Call Environment on Comfort With Class

Survey main effect 4.613 0.032

CC main effect 0.059 0.809

Survey x CC interaction 1.182 0.278

Figure 6. Self-reported comfort with class discussion

interest. As discussed above, there were no differences found between Hi

Hypothesis 3 deals with the effect of cold-calling on student comfort with

Survey main effect 0.26 0.612

CC main effect 0.15 0.698

Survey x CC interaction 4.93 0.027

Figure 7. Sensitivity analysis: Self-reported comfort with class discussion

above enables a post hoc comparison between cold-call environments in

no significant increase in comfort ratings for the Lo CC sections alone, t(239)

Discussion and Implications

students in high cold-calling sections are more comfortable participating in

Implications for Teaching

Limitations and Future Research

be an exploration of the relationship among specific questioning-and-

Furthermore, our study showed the longitudinal development of greater

Survey Variables Survey Questions