What Do the MCAs Measure?
Every spring, students across Minnesota sit for hours to take the Minnesota Comprehensive Assessments (MCAs) under the dutiful eyes of their teachers. Every summer, the results are released, and schools go into analysis mode, looking for areas of strength and areas for improvement. As teachers get ready to hunch over graphs and tables, we should remind ourselves of the bigger picture around the MCAs.
Students take math and reading MCAs in third and fourth grade, add science for fifth through eighth grades, detour to the GRAD Writing test in ninth grade, go back to reading in tenth, math in eleventh, and science whenever they finish a life science course in high school. Results can be broken down by grade, subject, sex, race, English Language Learner status, and reception of special education services. This generates a lot of numbers, and making sense of them can be challenging. While the temptation is to use test scores as a surrogate for the overall quality of education, it is worth considering what these scores directly measure, what they indirectly measure, what they fail to reliably measure, and what they do not measure at all.
What the MCAs Directly Measure
- How accurately students answered particular math, reading, and science* questions when they took the tests.
What the MCAs Indirectly Measure
- Broader student capabilites in math, reading, and science.
- The aggregate impact of past teacher and family support.
- Student investment in a test that—with the exceptions of the 10th grade reading and 11th grade math tests—has no impact on the rest of their lives.
- Students’ intrinsic motivation.
- Student test-taking skills.
- Student test-taking endurance—MCAs are typically administered in multi-hour blocks, spread over a few days in high school and a few weeks in younger grades.
- Student proficiency with the testing medium (believe it or not, students get different scores on the same test when it’s administered via computer or paper).
What the MCAs Fail to Reliably Measure
- Value added by teacher(s) during the preceding year.
- Overall quality of individual teachers.
What the MCAs Do Not Measure
- Untested skills in social studies, writing (beyond a basic level)*, world language, technology, or the arts.
- The responsibility of school factors versus external factors for student performance.
- At the secondary level, which teachers contributed in what ways to student scores.
- Student ability to address open-ended problems, work collaboratively, or create independently.
Therefore, while the MCAs have some value for educators, they are limited when diagnosing causes of performance. This is one reason why many districts have adapted nationally used diagnostic tools—like the adaptive testing offered by the Northwest Evaluation Association (NWEA)—for more regular, nuanced, and immediately applicable data gathering. Teachers can use more up-to-date information from these tests with greater ease than they can the once-a-year snapshot offered by the MCAs.
Standardized tests have a place in education, but that place is smaller than many suggest. If policymakers want to see teachers improve, the MCAs are administered too infrequently and offer too little diagnostic precision to help. For that goal, we’d be better off supporting fair evaluations that require a teacher to develop assessments validated by a trained observer, teach an observed lesson, and then analyze the data from their validated assessments with the observer to identify next steps for professional development.
If, on the other hand, the goal is to provide an evaluation of which schools are the most successful, we’d be better off developing mechanisms that evaluate students’ success after they graduate. That, after all, is the goal of our schools, and while such data would be more inconvenient to gather, it would paint a more detailed picture of a school’s ability to set its students on the right academic trajectory. (Even such a system, however, would struggle to disambiguate family effects from school effects or identify which teachers—or schools in the case of highly mobile students—most contributed to student outcomes.) At the very least, we must recognize both the strengths and the limitations of the MCAs, and we must demand a wider array of measurements be used when assessing our schools.
* The GRAD Writing test, administered in ninth grade, routinely has the highest passing rate in the state—generally in the 80-90% range—but it is also the only time the state assesses student writing. Listed below are the criteria for passing the test; whether the three-paragraph essay that could meet these is a sufficient guarantee of student success post-graduation is left as an exercise to the reader.
- Is related to the assigned topic.
- Has a central idea that is clearly expressed.
- Has some supporting details and sufficient development.
- Has a beginning, middle, and end.
- May present minor obstacles for the reader in moving from idea to idea.
- May have errors in sentence formation, word usage, and mechanics, but they do not substantially detract from the overall quality of the composition.