Research Summary
Decision-Making
Espin, C., Shin, J., Deno, S., Skare,
S., Robinson, S., & Benner, B. (2000). Identifying
Indicators of Written Expression Proficiency for Middle
School Students. (The Journal of Special Education,
34(3), 140-153.)
|
Short Summary of Article: CBM
scoring metrics used to evaluate written expression
at the elementary level are not valid for students
at the secondary level. Researchers examined the
reliability and validity of different measures of
written expression for students in 6th, 7th, and
8th grades. The best measure for secondary students
appears to be correct minus incorrect word
sequences. Percentage of words spelled correctly
and percentage of correct word sequences are valid
as screening tools but present problems when used
for progress monitoring. Duration of writing (3 or
5 minutes) and type of writing (descriptive or
story) did not affect the technical adequacy of the
writing samples.
|
Research at the elementary level has
supported the validity of three CBM scoring metrics, the
number of words written, the number of words spelled
correctly, and the number of correct word sequences, as
indicators of students' general performance in written
expression.
Research at the secondary level has indicated
that two of the CBM scoring metrics used at the
elementary level are not appropriate for the secondary
level. The number of words written and words spelled
correctly are not sensitive to group differences and
result in only low to moderate correlations with
criterion measures.
Tindal and Parker (1989) and others examined the use
of two percentage measures: the percentage of words
spelled correctly and the percentage of correct word
sequences. They found that the percentage measures were
good predictors of holistic ratings with correlations of
.73 and .75, respectively. In addition, the scoring
metrics significantly discriminated between students in
special and remedial education. They concluded that
percentage measures are valid for screening and
eligibility but present special difficulties for growth
monitoring. For example, suppose a student writes 20 word
sequences with 18 correct in the fall and 50 word
sequences with 40 correct in the spring. The student has
increased over the course of the year from writing 18 to
40 correct word sequences; however, the student's
percentage score has decreased from 90% to 80%.
A combination of scores shows promise for growth
monitoring, but it is not clear how a combination of
scores could be graphed to represent progress over
time.
Issues that have not been addressed in the secondary
school research include reliability, type of writing, and
duration of writing. The purpose of this study was to
investigate the reliability and validity of written
expression measures for middle school students.
Participants included 1000 students in grades 6, 7, and
8.
"The two research questions addressed in the study
were as follows: (1) What are the best quantitative
indicators of performance for use in the CBM of written
expression for middle school students? (2) What effect
does sample duration and type of writing have on the
selection of growth indicators?" p. 143
Results
(1) "Results of the study provide a basis for
concluding that correct minus incorrect word
sequences might serve as the best indicator of
students' general writing proficiency." p. 149
(2) The researchers compared results with two
different tasks: descriptive writing ("Describe the
clothing that students in your school wear.") and story
writing ("I stepped into a time machine and…). The
researchers also compared results with 3- and 5-minute
timed writing samples. Their results revealed very few
differences in technical characteristics related to type
of writing or duration.
Unfortunately, none of the computer-scored measures
(words written, words spelled correctly, characters,
sentence, characters per word, words per sentence) were
strongly related to the criterion measures. Researchers
have examined sentences written as a possible index of
writing and found that it was not sensitive to small
changes across time.
A weakness of the study relates to using the computer
for writing. The authors write,
"Whereas computer use does not seem to have a general
effect on students' writing, there is some indication
that computer use may differentially affect students'
writing on the basis of typing and word processing
skills." p. 151
