Professional Documents
Culture Documents
Art of Questioning and Test Construction Presentation-JSC
Art of Questioning and Test Construction Presentation-JSC
3. Application
Do you know of families like this one?
6. Evaluation
What are the advantages and disadvantages of living in
an extended family?
Illustrative examples from Let's Eat!, written by Ana Zamorano and illustrated by JulieVivas.
Taxonomy of Personal Engagement
(Morgan & Saxton, 1988)
1.
being curious about what is presented
Interest
2.
wanting to be, and being involved in the task
Engaging
3.
developing a sense of responsibility towards the task
Committing
4. merging objective concepts (the task or what is to be
Internalising learned) with subjective experience (what is already
owned) resulting in understanding and therefore
ownership of new ideas
5.
Interpreting wanting and needing to communicate that
understanding to others
Student Placement,
Diagnosis of Difficulties,
Checking Student Progress,
Reports to Student and Superiors,
Evaluation of Instruction.
Principles of Test Construction
For a psychological test to be acceptable it must
fulfill the following three criteria:
1. Standardization
2. Reliability
3. Validity
APP-79
Standardization
Standardizing a test involves administering the
test to a representative sample of future test
takers in order to establish a basis for
meaningful comparison.
Normal Curve
Standardized tests establish a normal distribution
of scores on a tested population in a bell-shaped
pattern called the normal curve.
Reliability
A test is reliable when it yields consistent results. To
establish reliability researchers establish different
procedures:
1. Split-half Reliability: Dividing the test into two
equal halves and assessing how consistent the
scores are.
2. Reliability using different tests: Using different
forms of the test to measure consistency between
them.
3. Test-Retest Reliability: Using the same test on two
occasions to measure consistency.
Validity
Validity of a test refers to what the test is supposed
to measure or predict.
1. Content Validity: Refers to the extent a test measures
your definition of the construct
2. Criterion-related validity: Relationship between scores
on a test and an independent measure of what the test
is supposed to measure
1. Predictive Validity: Refers to the function of a test in
predicting a particular behavior or trait. For instance, we
might theorize that a measure of math ability should be able
to predict how well a person will do in an engineering-based
profession.
2. Convergent Validity: We might correlate the scores on our
test with scores on other tests that purport to measure basic
math ability, where high correlations would be evidence of
convergent validity.
TEST PLAN (TABLE OF SPECIFICATIONS) TEMPLATE
(CERBIN, 2009)
A Test Plan (Table of Specifications) classifies each test item
according to what topic or concept it tests AND what objective it
addresses. The table can help you write a test that has content
validity—there is a match between what was taught and what is
tested. The table helps insure that you
1.emphasize the same content you emphasized in day-to-day instruction (e.g.,
more items about topic X and fewer about topic Y because you consider X to
be more important and you spent more time on X)
2.align test items with learning objectives (e.g., important topics might
include items that test interpretation, application, prediction, and
unimportant topics might be tested only with simpler recognition items)
3.do not overlook or underemphasize an area of content
You can create the Test Plan as you teach the class by inserting the
topics/concepts covered each day, and to the extent possible writing 1-2 test
items while the class period is still fresh in your mind.
TEST PLAN (TABLE OF SPECIFICATIONS) TEMPLATE
(CERBIN, 2009)
General Guidelines
Setting Questions
Rubric & Question Paper Format
Balance of Questions and Marks
Question Structure Checklist
Mark Scheme
GENERAL GUIDELINES
Check that
the mark scheme is complete for each question.
Check that
the mark scheme agrees with the question.
Guide questions:
• Does your question have everything it needs?
• Will your stem give away the answer?
• Are they all believable?
Checklist for Multiple-Choice Questions
numbers in order
alphabetize
or by length
Basic Form
Same basic principles as multiple-choice
basically just a bunch of m-c questions which share same
alternatives
the left column is called the "premise"
the right column is called the "option" or "response"
Advantages
same as Multiple Choice or True False except have
additional advantage of:
reduces guessing
--ten options = 1 chance out of 10, then 1 out of nine, etc.
-- if options can be reused = 1 out of 10 (or whatever) each time
Guidelines on Matching Questions
ADVANTAGES
• Easy to build
No guessing --> kid has to come up with answer rather than just recognize it
Actually faster to answer than mc because no alternatives to read through
Better diagnostic info (see where they go wrong)
DISADVANTAGES
Hard to write because they can fill the blank with unintended responses
Easier to bluff through ---> teachers hate giving filled in blanks zero
one remedy is to announce that irrelevant responses will get zero
could specify under blank, though usually unnecessary if carefully written
Essay Tests
Advantages
No guessing
have to generate answers rather than select them
students can demonstrate their knowledge within broad limits
Allows divergent thinkers to demonstrate originality, creativity
in contrast to multiple-choice which are terrible for divergent thinkers who always chose the
answer that isn't in the list of alternatives provided
of course, leaves open danger of diverging a bit too far...
Reduced lead time required to produce
less stuff to type, run-off if caught short
write 'em on the blackboard if you have to
Less work to administer for smaller number of students
easier to mark ten essays than to build a multiple-choice test for ten students; advantage
decreases with class size
but also have to consider long-term: teaching this course again next year?
Can be rich in diagnostic information
one topic in real depth
Essay Tests
Disadvantages
Impossible to mark objectively
Not valid measure of social or English if handwriting gets in the way
teacher expectations more important than actual performance
halo effects: student's earlier work prejudices expectations
Even different times of day make a difference
First paper to be read often sets standard, or if you decide later to lighten up,
the first paper's in trouble
Can sample only limited range of course CONTENT
Time consuming for student to write
Time consuming (and mind numbing) to mark
Certainly kills revision; Puts a heavy emphasis on student's writing, not thinking,
skills
poor writers are slaughtered, no matter how well they know content
Ultimately, have to evaluate writing or students won't learn to do it, but term paper may be better
approach to this
Only use essays when essays are called for
Increasing Objectivity of Essay Scoring