On May 5, 2017, the presidents and executive directors of the member societies of CBMS received a report from Samantha Burg and Stephen Provasnik at the U.S. Department of Education’s National Center for Education Statistics (NCES) on the results in mathematics from the 2015 studies by the National Assessment of Educational Progress (NAEP) and Trends in International Mathematics and Science Study (TIMSS). The full PowerPoint of their presentation, covering both the 2015 NAEP and 2015 TIMSS, can be accessed at www.cbmsweb.org/2017/05/presentations.
Both assessments are conducted for students at grades 4, 8 and 12. NAEP is a federally mandated assessment of student achievement in the U.S. and is conducted every other year. TIMSS provides an international comparison and is run every four years for ages equivalent to grades 4 and 8. The 12th grade TIMSS is restricted to advanced mathematics students (in the U.S. those who have taken a course like AP Calculus). It was administered in 2015 for the first time in the U.S. since 1995.
The scores since 1990 for the 4 th and 8 th grade NAEP and since 2005 for the 12th grade are shown in Figures 1, 2, and 3. The distinguishing features for grades 4 and 8 are the strong growth from 1990 until 2007 and relative stagnation since then, with a small but statistically significant drop (except for the 90th percentile in grade 4) between 2013 and 2015. The 12th grade scores also show a drop since 2013 that is statistically significant at and below the 50th percentile.
This drop is a cause for concern, but not yet alarm. NCES is eagerly anticipating the 2017 NAEP results to see whether the downturn was simply a blip in what is essentially a stable state or the start of something more troubling.
 |
| Figure 1: NAEP scores for grade 4. Source: Burg & Provasnik, 2017. |
 |
| Figure 2: NAEP scores for grade 8. Source: Burg & Provasnik, 2017. |
 |
| Figure 3: NAEP scores for grade 12. Source: Burg & Provasnik, 2017. |
An obvious question is whether the Common Core State Standards in Mathematics (CCSS-M) have had any effect on student scores. One hypothesis is that changing the curriculum has introduced enough confusion and uncertainty among teachers that it is having a visibly negative effect. Another hypothesis, which has some supporting evidence, is that the choices of topics for assessment may no longer be completely aligned with what is being taught.
The largest drops at Grade 4 were in the subject areas of Geometry and Data Analysis (Table 1). The NAEP Validity Studies (NVS) panel (Daro, Hughes, & Stancavage, 2015) found some misalignment between the NAEP questions and the CCSS-M curriculum. They found that 32% of the Data Analysis questions were either not covered in CCSS-M or were covered after grade 4. In Geometry, 18%, of the NAEP questions were covered after grade 4 in CCSS-M. In the other direction, only 57% of CCSS-M standards for Operations and Algebraic Thinking by grade 4 were covered by NAEP questions.
 |
| Table 1: Changes in NAEP scores, 2013 to 2015, by subscale topics. Source: Burg & Provasnik, 2017 |
For grade 8, the misalignment occurs in both directions within Data Analysis. In the 8 th grade NAEP, 17% of the Data Analysis questions had not yet been covered in CCSS-M, and 59% of what is specified for statistics and probability by grade 8 in CCSS-M was not assessed by NAEP. For grade 12, there was a uniform 2-point drop across all subscales.
These observations raise interesting questions about the construction of future NAEP instruments. Because of the need for comparability from one test administration to the next, the distribution of topics has not changed. While CCSS-M is not the national curriculum that was once envisioned, the fact is that almost all states have aligned their standards with its expectations. NAEP may need to change to reflect the reality of what is taught by grades 4 and 8.
The breakdowns by race/ethnicity and gender for the overall mathematics scores in grades 4 and 8 (Table 2) show comparable increases from 1990 to 2015, and comparable declines since 2013. Black students in grade 4 saw the greatest gains since 1990, but at a score of 224 they are still well below the national average.
 |
| Table 2: Changes in NAEP Math scores for grades 4 and 8 by race/ethnicity and gender. Source: Burg & Provasnik, 2017. |
At grade 12, the strongest gains since 2005 have been for Asian and Hispanic students (Table 3, Pacific Islanders are such a small proportion of Asian/Pacific Islander that it is not clear how their scores have changed, and the doubling of the percentage identifying as Two or More Races makes it difficult to compare the 2005 and 2015 scores). An interesting insight lies in the shift in the demographics of 12 th grade students. In ten years, the percentage of White students dropped from 66% to 55%, while the percentage of Hispanic 12 th graders rose from 13% to 22%.
 |
| Table 3: Changes in NAEP Math scores for grade 12 by race/ethnicity and gender. Source: Burg & Provasnik, 2017. |
Next month I will be looking at the changing demographics of bachelor’s degrees earned in engineering, the mathematical sciences, and the physical sciences. In mathematics, the decline in the percentage of degrees in mathematics going to White students has been in line with the decline in their overall percentage at that age group, from 72.4% in 2005 to 59.6% in 2015 (NCES, 2005–2015). Some of this has been made up by a significant increase in mathematics degrees going to Hispanic students, from 5.7% to 8.9%, but the percentage of bachelor’s degrees in mathematics earned by Black students decreased from 6.1% to 4.7% over this decade, while Asian students remained essentially stable, 10.2% to 10.6%. Most of the shift has gone to non- resident aliens who accounted for 5.0% of the mathematics degrees in 2005, but 12.9% in 2015.
References
Burg, S. & Provasnik, S. (2017). NAEP and TIMSS Mathematics 2015. Presentation to the Conference Board of the Mathematical Sciences, May 5, 2017. Available at www.cbmsweb.org/2017/05/presentations/
Daro, P., Hughes, G.B., & Stancavage, F. (2015). Study of the alignment of the 2015 NAEP mathematics items at grades 4 and 8 to the Common Core State Standards (CCSS) for Mathematics. NAEP Validity Studies Panel report. Washington, DC: American Institutes for Research. Available at www.air.org/sites/default/files/downloads/report/Study-of- Alignment-NAEP-Mathematics- Items-common- core-Nov- 2015.pdf
National Center for Education Statistics (NCES). (2005–2015). Digest of Education Statistics. Available at nces.ed.gov/programs/digest/
Note:
In compliance with new standards from the U.S. Office of Management and Budget for collecting and reporting data on race/ethnicity, additional information was collected beginning in 2011 so that results could be reported separately for Asian students, Native Hawaiian/Other Pacific Islander students, and students identifying with two or more races. In earlier assessment years, results for Asian and Native Hawaiian/Other Pacific Islander students were combined into a single Asian/Pacific Islander category.
As of 2011, all of the students participating in NAEP are identified as one of the following seven racial/ethnic categories:
- White
- Black (includes African American)
- Hispanic (includes Latino)
- Asian
- Native Hawaiian/Other Pacific Islander
- American Indian/Alaska Native
- Two or more races
