|
Heartland AEA 11
6500 Corporate Drive
Johnston, IA 50131
515/270-9030
800/362-2720 |
|
Research Summary
Decision-Making
Helwig, R., & Tindal, G. (2003). An experimental analysis of
accommodation decisions on large-scale mathematics tests.
Exceptional Children, 69, 211-225.
The authors defined a testing accommodation as
"a change in test presentation or response format that does not
alter the construct under consideration" (p. 211). The potential
consequences of assigning certain accommodations to students is
reviewed briefly (e.g., expenditure of valuable resources, affect
the validity of inferences made from test results, reading aloud is
frustrating and distracting to some students). The literature on
teacher effectiveness at assigning accommodations and the profile
of students who benefit from certain accommodations also is
reviewed briefly.
At total of 1550 students participated in this
study, but only 1218 of those students completed all of the
measures. Data analyses occurred for 245 students receiving special
education. Of those students in special education, the majority of
the students were identified as having a learning disability. The
general and special education teachers providing math instruction
to the participating students also participated. Participants came
from elementary (grades 4 and 5) and middle schools (grades 7 and
8) from eight states.
Four dependent measures were used. First, two
30-item, multiple choice mathematics achievement tests (Form A and
Form B) were created from a seventh-grade item bank. Items were
reviewed by each state to ensure the items were congruent with
their state's curriculum. Each form (Form A and Form B) was created
in two formats. The first format was standard and had items
presented in written form in a test booklet with several items per
page. The second was a video format. The video included the face of
an actor who read each item on one portion of the screen while the
text of the problem was shown on the remaining space. The test
booklet for this form contained one item per two facing pages.
Students bubbled their answers on a separate answer sheet in both
formats. Second, the teacher who taught the participating student
also completed a survey on the student's skill level in both
reading and mathematics. The third measure used was a math skills
test, which all students took. The test was the same for fourth-
and fifth-grade students; students in seventh and eighth grade took
a different math skills test. The fourth measure was a reading maze
designed to measure reading proficiency.
Classrooms of students were assigned randomly to
one of two groups. Both groups took each form (Form A and Form B)
with 1- to 4-day intervals between administrations. The order of
administration was counterbalanced within each group.
A magnitude of the difference (Diff) was
calculated for each group (Group 1, Group 2, elementary students,
middle school students) by subtracting the paper-and-pencil z score
from the accommodated z score, thus obtaining the relative
performance between the two formats in standard deviation units. A
positive number indicated the strongest performance had occurred
with the oral presentation. The majority of the data analysis is
reported for only those students within special education who
experienced a .5 magnitude shift.
Nearly 600, or 46%, of the students experienced
a .5 magnitude shift. Of the 122 special education students, 62
favored the standard version and 60 favored the video format.
Teachers made incorrect recommendations (i.e., teachers felt it was
important for them to receive a test accommodation to increase test
performance) for 52% of the elementary students and 50% of middle
school students. Teachers, by rating that an accommodation was not
important, made incorrect recommendations for 14% of elementary
students and 88% of middle schools students. Overall,
accommodations for 45% of elementary students and 57% of middle
school students were incorrectly recommended by their
teachers.
Overall, the teachers in this study did not
appear to make accurate recommendations of which students would, or
would not, benefit from having math test items read aloud. That is,
teachers tended to over-recommend accommodations. The authors
hypothesized that students with low reading skills but adequate
math skills would be the most likely to benefit from the read-aloud
accommodation. However, the low readers with adequate math skills,
at both the elementary and middle school, performed better on the
standard format, not the video format.
The authors suggest implementing an
accommodation within the classroom prior to testing to reduce any
novelty effect and possibly increase teacher accuracy of assigning
accommodations.
|
|
