Professional Documents
Culture Documents
Using Cloze Tests PDF
Using Cloze Tests PDF
Linda Todd
Introduction
The four tests contained in this booklet were devised and trialled by the author as a
research project for MA in Applied Linguistics in 2006. Tests were devised to provide a
valid and reliable measure of assessing EAL students reading skills to inform and monitor
learning programmes.
Acknowledgements
The author would like to thank the following people who made this publication possible:
My academic supervisor, Dr Peter Gu
The teachers at the schools where the tests where trialled; Lisa Lorenzen Jenny Olsen,
Rosie Salas, Barbara Slevin, Duong Tran and Diane Smithson
Special thanks to Mary Busch for her guidance in writing this as a publication suitable for
teachers.
Linda Todd
Wellington
2008
Cover photo: Faduma Salad and Thary Trann, students from Wellington East Girls
College, on a trip to Kapiti Island.
2
Using cloze tests to assess the reading needs of EAL learners
Contents
Rationale 4
These tests
Scoring procedures 5
What can the test scores tell us about the literacy proficiency of learners?
Exemplars 6
References 12
Stage 2 17
Stage 3 20
Stage 4 23
3
Using cloze tests to assess the reading needs of EAL learners
Rationale
Cloze tests, where deletions have been specifically chosen, produce diagnostic
information on a students integrated language skills. A reading passage must not be too
difficult for a learner and be on a culturally familiar topic for a learner to engage with the
text (Douglas 1998; Brown 2001; Sasaki 2002). Cloze tests are relatively easy to
administer and provide valid and reliable information about a students language
proficiency by deliberate deletion of targeted language features such as verb forms,
prepositions and lexical items. (See Research notes.)
These tests
The four tests contained in this booklet were devised and trialled by the author as a
research project for MA in Applied Linguistics in 2006. Deletions are specifically selected
to test a range of linguistic knowledge including lexical and grammatical structures. Each
test contains a range of deletions that are solvable and no deletions require complex
technical knowledge. The tests follow an increasing level of complexity based on reading
stages 1, 2, 3 and 4 of the English Language Learning Progressions (ELLP).
The results provide a clear indication of a learners ability to interpret text at a particular
level. Close analysis of a learners gapfill choices can reveal information about their
vocabulary knowledge, knowledge of grammatical structures and syntax, and their ability
to understand text without prior teaching. Analysis of results should provide a guide for
programme planning for the learner and be used with other information about their
proficiency and progress in all language skills, for example results from Nations
Vocabulary Survey, Probe tests, asTTle, ELLP writing and oral language assessments.
4
Using cloze tests to assess the reading needs of EAL learners
Scoring procedures
There are 45 or 50 items to mark for each test. These should be converted to a
percentage.
Accept as correct answers that indicate the learner has understood the context even
if there are errors in spelling or minor grammatical errors.
Be consistent when marking a group or class and decide whether you will accept
switches in tense, incorrect pronouns, article or prepositional errors, or errors in verb
form or word form choices.
In the research study that trialled these tests the following practice was consistently
applied:
missing s on third person singular present tense verbs but eat insects was not
accepted for eating insects (test 2.3)
missing s for plurals was accepted
some leniency was allowed in prepositional use, e.g. in, around was accepted for
on the islands; but for in the bush, on the bush was not acceptable (test 2.2);
between 2001 and 2005 to was not accepted, although commonly used by the
students (test 1.4). However, in test 3.1 there were two contrasted headings, the first:
Some ways to save water in the home (followed by inside examples); the second
heading: Some ways to save water outside the home (followed by outside
examples), in was not accepted for the latter example
article usage: an award, a or the were accepted (test 1.4), other article errors
(the/a) were accepted in most cases
pronoun concordance: our lives where there were two instances of we in the same
paragraph, your or their were not accepted (test 3.2)
spelling was allowed considerable leniency as long as the target word was
understood, then was accepted in the place of more than eight, off course was
accepted for of course (test 3.2)
punctuation: capital letters at the start of sentences, frequently missed by many
students, was deemed acceptable
subject verb concordance was allowed some leniency, A Hindu do not eat beef
(test 2.2) was deemed acceptable.
What can the scores tell us about the literacy proficiency of the
learners?
Oller (1979) established the following guide for the readability of a text:
Scoring below 44%, a learner is likely to experience frustration
Scores between 44 and 57%, a text may be suitable for instructional reading use.
Scores of 58 to 100% should allow a learner an increasing level of independence in
reading the text.
5
Using cloze tests to assess the reading needs of EAL learners
Exemplar showing the application of the marking guide to a passage
from Stage 3
(1) Should The number of (1) water in the Wellington Region is Should be
be homes growing. (2)_that__ means more people are going to need or use (3)
or houses (3)_sea_ water (ie higher demand). More demand and (4) less (4)
rain water could lead to water shortages. machine (8)
them (9)
Some ways to (5)_save_ water in the home:
Dont leave the (6)_water_ running while you wash
(10) Should hands, dishes (7)_and_ vegetables. (12) Should be
be less if Only use the washing (8)_hand__ or dishwasher for full brush
learner has loads, never use (9)_water_for half loads. (A full load than
engaged with uses (10)_more__ water than 2 half loads!) Phonetic
context Turn the tap (11)_off_ after you have wet your (12) spelling was
prash_ when cleaning your teeth. acceptable
Take short showers (less (13)_then_ 5 minutes) rather
(14) Should be
than deep baths. (16) Should be
for but is
acceptable in
Check (14)_your__ leaks in water pipes around the outside
this context (15)_home_. in is not
acceptable in
Some ways to save water (16)_in_ the home the context of
Turn outside taps off properly (17)_after_check for the points
leaks. following.
(20) Should
be not if Use a bucket of (18)_water__ to wash cars instead of
learner has leaving the hose (19)__on_. (17) Should be
engaged Sweep the path with a broom, (20)_or__ a hose. and
with context Water plants in the cool of the day, (21)_once__ or
twice a week in summer. (Up to 80% of (22)_rain__
water used during the heat of the (23)_summer__ (22 )Should be tap
could evaporate.) or the and (23)
(24) Should day or afternoon,
be can
Mulch around plants to prevent evaporation (mulch
(24)_plants__ prevent up to 70% of water loss from but rain and
summer
evaporation). Mulch can (25)_be_ made from used
acceptable as they
paper, old carpet or garden wastes. make sense
(15/25 = 60%)
6
Using cloze tests to assess the reading needs of EAL learners
Exemplar of the marked text extract from Stage 3 demonstrating
corrections and possible student reading strategies
7
Using cloze tests to assess the reading needs of EAL learners
Analysing the scores
What can we learn about the learners reading strategies?
A cloze tests a range of reading skills that are integrated with one another. The level of
difficulty of the passage and the learners background knowledge of the topic play a big
part in determining their level of engagement and consequently their ability to fill the gaps
correctly. Learners who are not familiar with the cloze format may not have developed
strategies for solving gaps.
All learners use top down/bottom up processing. That is they apply their overall
understanding of the topic as well as their knowledge of vocabulary (particularly the
collocations of associated words), grammar and sentence structure.
More capable readers respond to cues such as headings, the initial sentence or
paragraph. They read around items and are not put off by unknown vocabulary. They
check their gapfill choices as they go.
Less capable readers become much more clause bound and choose words that
collocate with the word prior to or immediately past the gap. They may appear to apply
a scatter gun approach selecting words from the word bank that make no sense in the
context if checked. They frequently ignore punctuation, even the use of capital letters
at the beginning of a sentence.
control of vocabulary (word choices made correctly or incorrectly, did the student use
their own word or choose from the word bank?)
grammatical knowledge ( consistent use of tense, pronouns, correct choices based on
parts of speech, observation of plural, subject verb/ agreement)
observance of punctuation
accuracy of spelling
ability to write neatly, form letters correctly.
Make observation notes on the learners responses under the categories listed above.
See next page.
8
Using cloze tests to assess the reading needs of EAL learners
Exemplar of a learners response illustrating language strategies
(20) Does not drink alcohol, vegans eat only (18)_fruit_ from plants and a
observe
vegetarian (19) eat_ food from plants, but also eggs, (20) (18)Observes
punctuation nor
context clues,
read past the and well as cheese, yoghurt and (21)other_ products.
gap draws on
background
knowledge
(21) Uses Buddhists, Jews, Hindus and Muslims (22) _does_ not to
preceding eat certain foods for (23) _there_ reasons. A Buddhist does
context to make (19 ), subject verb
(24) _not_ kill anything to eat. A Hindu (25) do_ not eat agreement error
a sensible
choice
beef or pork. (26) _Also_ Jew does not eat pork or shellfish.
(22) Verb
(24)Observes (27) _Muslim_, Jewish people do not mix milk products (28) choice error,
context clues, present
draws on and_ meat. They eat kosher food. A Muslim does not (29)
tense +
background _eat_ pork, or drink alcohol. They eat halal food. infinitive
knowledge
9
Using cloze tests to assess the reading needs of EAL learners
After the test
Conference learners when returning the marked tests. Question learners on some of
their choices to more fully understand the strategies they were applying to the gap
filling process and their engagement with the text as a whole. With the learner, identify
key areas that they need to work on.
Use test scores as a contributor to an individual and class profile. Administer a higher
level test later in the year to track progress.
Use test results to help to establish approximate ELLP reading levels. For example,
Test Stage 3 is at ELLP Stage 3 and within the range of English Language Intensive
Programme (ELIP) Stage 3.
Make deletions to test a variety of language skills. The tests in this publication were
designed to ensure there was a balance of items that could be solvable by reference:
1. within the clause
2. outside the clause within the sentence
3. across sentences
4. outside the text, (to a learners background knowledge)
Bachman (1985)
Note the distribution of vocabulary levels in your text. You can do this electronically
using the Compleat Lexical Tutor website as follows:
10
Using cloze tests to assess the reading needs of EAL learners
The Compleat Lexical Tutor website also has the facility to generate cloze tests for you as
long as your text is saved as a text file. Go to the cloze generator which is accessible from
the home page. At http://www.lextutor.ca/cloze/n_word/ select the rational deletion option.
The method that works most successfully at the time of writing was:
Please note that this website is constantly being developed. One advantage of using this
tool is the fact that you can also generate the cloze to be done online so that a learner can
work independently and has immediate feedback on his or her word choices. There is
also scope to insert an audio or video file option.
Further studies were carried out that demonstrated cloze items do not carry the same
amount or kind of information. It was found that deleting specific items such as cohesive
devices, function words or lexical items required test-takers to move outside the clause
and engage higher level strategies (Bachman 1982,1985; Jonz 1990,1991; Cohen 1998;
Storey 1997; Yamashita 2003). The tests became known as rational cloze or gap-fill tests
(Alderson, 2000).
If reading passages are too difficult, learners cant activate their background knowledge
and become clause dependent (Douglas, 1998). Therefore it is important to ensure a
passage is of an appropriate level of difficulty for the learners being tested (Brown, 2001)
and that it contains culturally familiar content (Sasaki, 2000).
Further information regarding this research can be obtained by reading Todd and Gu,
(2007, p16-29).
11
Using cloze tests to assess the reading needs of EAL learners
References
Alderson, J. C. (1979a). The cloze procedure and proficiency in English as a foreign
language. TESOL Quarterly 13(2), 21927.
Bachman, L. F. (1982). The trait structure of cloze test scores. TESOL Quarterly 16(1),
6170.
Bachman, L. F. (1985). Performance on cloze tests with fixed ratio and rational deletions.
TESOL Quarterly 19(3), 53556.
Brown, J. D., Yamashiro, A. D., & Ogane, E. (2001). The emperors new cloze: strategies
for revising cloze tests. In T. Hudson & J. D. Brown (Eds.), A focus on language test
development: Expanding the language proficiency construct across a variety of tests (pp.
143-161)., Honolulu, HI: University of HawaiI Press.
Jonz, J. (1990). Another turn in the conversation: what does cloze measure? TESOL
Quarterly 24(1), 6183.
Jonz, J. (1991). Cloze item types and second language comprehension. Language
Testing, 8(1), pp. 122.
Oller, J. R. (1973), Cloze tests of language proficiency and what they measure. Language
Learning 23(1), 105118.
Oller, J.R. (1979), Language tests at school: a pragmatic approach, London, Longman.
Sasaki, M. (2000). Effects of cultural schemata on students test taking processes for
cloze tests: a multiple data source approach. Language Testing, 17(1), 85114.
Storey, P. (1997). Examining the test taking process: a cognitive perspective on the
discourse cloze test. Language Testing 14(2), 214231
Todd, L., & Gu, P. (2007). Rational cloze tests as a placement measure for EAL learners.
The TESOLANZ Journal 15, 16-29.
Yamashita, J. (2003). Processes of taking a gap-filling test: comparison of skilled and less
skilled EFL readers. Language Testing 20(3), 267293.
12
Using cloze tests to assess the reading needs of EAL learners
Stage 1:
Name: Date:
In the following four passages some of the words have been left out.
First read over the whole passage and try to understand what it is about.
Then try to fill in the spaces. It takes only one word to fill in each space.
You may use the words in the text box at the bottom of the page to help you. Any word
that makes sense in the space will be accepted.
There are seven continents in the world Europe, Asia, North America, South America,
Africa, Australia and Antarctica. There are also ________ large
oceans Pacific, Atlantic, Indian and Arctic.
Asia is the largest ________ and Australia is the smallest. Africa has the ________
countries. It has 53 countries. Europe is a ________ continent with 25 countries. There
are 22 ________ in North America and 13 countries in ________ America. Australia is a
continent, a country ________ an island.
Adapted from Ministry of Education (2003c). The World. English Language Intensive Programme
Years 713 Resource. Auckland: Ministry of Education. Stage 1, Reading.
13
Using cloze tests to assess the reading needs of EAL learners
1.2 Antarctica
Antarctica is the coldest, windiest and loneliest of all the seven continents. It also has high
________.
Temperature
In the coldest parts of Antarctica the ________ is usually between -40 and -70 Celsius
________ the winter.
Amount of daylight
In Antarctica, day and ________ are different from what we ________ used to. During the
coldest time, ________ is dark all day as well as ________ night.
Ice shelf
Antarctica is so ________ that some of the oceans around ________ freeze as well. This
is called an ________ shelf. Sometimes bits ________ the ice shelf break off. ________
are called icebergs.
Adapted from Ministry of Education (2003c). The World. English Language Intensive Programme
Years 713 Resource. Auckland: Ministry of Education. Stage 2, Reading.
14
Using cloze tests to assess the reading needs of EAL learners
1.3 Sharks
Sharks are fish. They belong to the group called Elasmobranchii. ________ live in waters
all around the ________. They live in every ocean and ________ some rivers and lakes.
Other ________ have bones but sharks dont. Their skeletons ________ made from
cartilage which is not ________ hard as bone.
Sharks have about 5 rows of ________. The front teeth are the biggest. Sharks ________
chew most food. They eat ________ in big pieces. Many people are ________ of sharks
because of their ________ teeth. But sharks dont usually attack ________.
All sharks are carnivorous (meat eaters) and they may ________ fish, squid, other sharks,
marine mammals ________ other small animals.
Adapted from Ministry of Education (2003c). The World. English Language Intensive Programme
Years 713 Resource. Auckland: Ministry of Education. Stage 2, Oral.
15
Using cloze tests to assess the reading needs of EAL learners
1.4 Tana Umaga
Tana Umaga is a famous New Zealand rugby player. ________ was born in Lower Hutt in
1973. His parents came to New Zealand from Samoa.
In 1994 Tana first ________ for Wellington in the NPC competition. ________ was
captain of the Hurricanes between 2001 ________ 2005.
He was first picked to play ________ the highest rugby team, the All ________, in 1997.
In 2003 he was given ________ award for outstanding sportsmanship. Umaga was made
captain of the ________ Blacks in 2004.
In 2006 Umaga decided ________ to play for the All Blacks ________ more. He wanted
to spend more ________ with his family. He still plays ________.
/45
16
Using cloze tests to assess the reading needs of EAL learners
Stage 2:
Name: Date:
In the following three passages some of the words have been left out.
First read over the whole passage and try to understand what it is about.
Then try to fill in the spaces. It takes only one word to fill in each space.
You may use the words in the text box at the bottom of the page to help you. Any word
that makes sense in the space will be accepted.
Wherever people live together in a community, there are many different jobs to be done.
People work to produce and sell goods or to provide services. Some people, however,
must do dangerous jobs to help a community run smoothly.
Some people face danger in their ________ every day, but they take ________ to
minimise the risks ________ face.
________ workers, such as police officers and fire fighters, ________ dangerous jobs to
help people ________ are in trouble.
Other workers, such ________ lion tamers, jockeys and racing car ________, do
dangerous jobs because they ________ the excitement and the thrill. ________ do
dangerous jobs because they are paid ________ to do their jobs. People may do
________ jobs for a combination of reasons.
17
Using cloze tests to assess the reading needs of EAL learners
2.2 What do you eat?
Some people eat everything. Some people do not eat everything. They will not eat some
types of food because of what they believe, or because of their religion. Who are these
people?
The ________who choose what they eat or ________ not eat because of ________
beliefs include teetotallers, vegans and vegetarians. ________ do not drink alcohol,
vegans eat only ________ from plants and a vegetarian ________food from plants, but
also eggs, ________ well as cheese, yoghurt and ________ products.
Buddhists, Jews, Hindus and Muslim ________ not to eat certain foods for ________
reasons. A Buddhist does ________ kill anything to eat. A Hindu ________ not eat beef or
pork. ________ Jew does not eat pork or shellfish. ________, Jewish people do not mix
milk products ________ meat. They eat kosher food. A Muslim does not ________ pork,
or drink alcohol. They eat halal food.
Glossary:
Adapted from Denny, H. ed. (2005). What Do You Eat? Password. Volume 29, Number 3.
18
Using cloze tests to assess the reading needs of EAL learners
2.3 Wax eyes
About a hundred and thirty years ago, people at Waikanae, north of Wellington,
saw great numbers of strange birds.
They were little green birds ________a white ring of feathers round ________ eyes.
Scientists soon found out that these ________had come all the way ________ the
Tasman Sea by very strong ________.
The Maoris called the new bird tauhou, their ________for stranger. Today they are
also known ________wax-eyes, white-eyes, or silver-eyes.
Because they can ________ all sorts of food, and ________they can live in many different
________of places, wax-eyes have ________ in numbers. They are now found ________
over New Zealand along the coasts, ________the islands, in scrub, in tussock country,
________the bush, on farms and in city ________. There are probably more wax-eyes in
New Zealand ________any other kind of bird.
Usually, you ________them in groups, hopping very ________ through bushes and trees.
They ________ a high twittering noise most of the time as they work along, ________
insects, nectar and berries. Their ________may help them to recognise each other and to
keep the group together.
Adapted from Brockie, R.E. (1981). Wax-eyes. School Journal. Wellington: Learning Media. Part
2, Number 4.
/50
19
Using cloze tests to assess the reading needs of EAL learners
Stage 3:
Name: Date:
In the following two passages some of the words have been left out.
First read over the whole passage and try to understand what it is about.
Then try to fill in the spaces. It takes only one word to fill in each space.
You may use the words in the text box at the bottom of the page to help you. Any word
that makes sense in the space will be accepted.
20
Using cloze tests to assess the reading needs of EAL learners
3.1 Why save water?
We are having longer, drier, hotter summers. This means there will be less water to go
around. It also means people will use more water to keep cool and keep their gardens
growing.
The number of ________ in the Wellington Region is growing. ________ means more
people are going to ________ water (i.e. higher demand). More demand and ________
water could lead to water shortages.
21
Using cloze tests to assess the reading needs of EAL learners
3.2 Are you getting enough sleep?
Nothing is worse than lying awake hour after hour, night after night, when all you really
want to do is sleep.
Insomnia is the ________ used to describe the habit of sleeplessness. For some
unlucky people, sleeping difficulties may continue not just for days, but for weeks, or even
________.
Being overtired can have a very ________ effect on every part of ________ lives. We
may get angry easily ________ often marriages and friendships suffer. ________ can
make us unhealthy and ________. We may catch colds easily or ________ more serious
sickness. Our work or ________ suffers. Our judgement suffers many fatal accidents
are ________ by drivers falling asleep at the ________ of a car.
How much sleep ________ adults need? Most of us ________ about eight hours sleep a
night, although, ________ course, some people need more and ________ need less.
Children and teenagers need ________ ten hours. If you have a few ________ in a row of
sleeping less ________ eight hours, you should try to ________ up for this by sleeping
more ________ you get the chance. A 1959 American Cancer study ________ one million
Americans showed that ________ who sleep about eight hours a night ________ likely to
live longer than those ________ sleep less than seven hours. ________ studies agree
with these findings.
Adapted from Hallam, E and Jones, C. (2003) Are You Getting Enough Sleep? Practice Makes
Perfect 1. Auckland: New House Publishers.
/50
22
Using cloze tests to assess the reading needs of EAL learners
Stage 4:
Name: Date:
In the following three passages some of the words have been left out.
First read over the whole passage and try to understand what it is about.
Then try to fill in the spaces. It takes only one word to fill in each space.
You may use the words in the text box at the bottom of the page to help you. Any word
that makes sense in the space will be accepted.
Dont you just hate it when you feel awkward and your face turns red? Its as though
the whole world ______ exactly how youre feeling without you ______ them!
Blushing usually happens when youre around ______ people because a group
situation makes ______ more sensitive to whats being ______ or done. Sometimes
embarrassment causes the ______. Sometimes not telling the truth can ______ you
blush which can be very _______for the person youre talking ______ will have a
pretty good idea that ______ not telling things the way ______ really are!
23
Using cloze tests to assess the reading needs of EAL learners
4.2 Money Trees
At last, scientists have been able to confirm that money really does grow on trees.
After six years of arduous research, Chris Anderson at Massey University emerged
from his experimental greenhouse with a range of fast-growing herbs that can
________ minute gold particles from the soil.
Using a garden-variety mustard, Dr Anderson has been able to show that the
________ can pull tiny gold particles from the gold-________ soil, along with the
other soil-bound ________ they need to grow. So far the tests have been ________
on a small scale, _________ with a field trial in Brazil ________ this year. The next
step is to do full-________ field trials.
Adapted from Author unknown (2003) Money trees New Zealand Education Gazette, 6
October, Page 14. In Ministry of Education (2008). The English Language Learning
Progressions: Years 913. Wellington: Learning Media.
24
Using cloze tests to assess the reading needs of EAL learners
4.3 The Gene Seekers
Years of tragic deaths from stomach cancer brought members of a Bay of Plenty
wh nau, medical specialists and genetic scientists together to seek a solution.
Hira had seen eight of ______ brothers and sisters die from ______ cancer. In the
space of ______ a few years, I ______ five brothers, three sisters, and several nieces
and nephews. And one of ______was only sixteen years old.
Not ______ was the personal loss ______ to bear, so too was the fear the ______
constantly carried. We were ______ thinking the worst, says Hira. ______ would
think - who is ______? Will it be my brother, my ______, or me? Every pain you get,
you think ______ it. And you become too frightened to go to the ______.
Rates of stomach cancer differ around the ______, but in New Zealand, the chances
of ______ from this disease ______ just over 6 per 100 000 for ______ of European
descent. In the Maori and Pacific Islands ______, the rate is ______ at 22 per 100
000. However, in Hiras whnau in the Bay of Plenty, the incidence was very much
______. Worse still was the ______ onset of ______ often fatal disease. Normally,
stomach cancer strikes people ______ the age of sixty, but in the whnau, it was
common for people in their twenties and thirties to be______. Even teenagers were
dying from the cancer.
Glossary:
whnau = extended family
Adapted from OBrien, B. (2001). The Gene Seekers. Wellington: Learning Media.
/50
25
Using cloze tests to assess the reading needs of EAL learners