MAA Calculus Study: Effects of Calculus in High School

This month I return to the MAA Calculus Study, Characteristicsof Successful Programs in College Calculus, with a report on what we learned about the effects of taking calculus in high school. Because this study only looked at students in Calculus I, we can say nothing about how many of these students never take another calculus class or how many start their college mathematics with Calculus II or higher. Since the survey was conducted in the fall term, we cannot even say anything about students who might postpone taking Calculus I until later in the academic year. But, thanks to this survey, we can say a lot about the students who study calculus in high school and then begin with Calculus I in their first term at college.

First of all, we have a good idea of how many students this involves. Of the 300,000 who enrolled in Calculus I in fall 2010, just over half had studied calculus in high school. That represents just about one quarter of the 600,000 or so students who had studied calculus in high school the previous year. Of those who had taken the AP Calculus exam that spring, just over a quarter were in Calculus I that fall. From the latest CBMS report [1], about 55,000 incoming freshman arrived with credit for Calculus I. This leaves almost two-thirds of those who studied calculus in high school neither acquiring college credit for their high school work nor enrolling in Calculus I in a fall term.

We can break these data down further by score on the AP exam:

• Half of those who scored a 1 or 2 on the BC exam or a 3 on the AB exam enrolled in Calculus I in the fall.
• A third of those who scored a 3 on the BC exam enrolled in Calculus I.
• A quarter of those who scored 1, 2, or 4 on the AB exam enrolled in Calculus I.
• 1 in 8 of those who scored a 5 on AB or a 4 on BC enrolled in Calculus I.
• 1 in 20 of those who scored a 5 on BC enrolled in Calculus I.
• A quarter of those who scored a 3 or higher on the AB or BC exam received college credit for Calculus I.

Of those who arrived with credit for Calculus I, we do not know how many used it to place into a higher calculus class and how many never studied any further calculus in college.

At the research universities, those characterized by offering a doctorate in mathematics and dominated by the flagship state universities, over 70% of the students in Calculus I had completed a course of calculus in high school. We have data for about 5,000 of these students and so can report fairly accurately on the effect of studying calculus in high school on student performance in Calculus I in college. As shown in Figure 1, less than a third studied no calculus in high school, less than a third studied calculus but did not take the AP Calculus exam, a third took the AB exam, and a small but significant percentage (8%) took the BC exam.

 

Figure 1: Distribution of high school calculus experience among Calculus I students at research universities.

There is a common perception among students that having studied calculus in high school gives students a significant advantage in Calculus I when they get to college. Our data tend to confirm that. Figure 2 compares the final grades of the total student population at research universities with three subsets: those who did not study calculus in high school, those who did and did not take the AP Calculus exam, and those who did and did take the AP Calculus exam. We see a much lower percentage of A’s and a much higher percentage of DFW’s (grades of D or F or a withdrawal from the course) among students who did not study calculus in high school as opposed to those who did. There is little difference in Calculus I grades between those who did and those did not take the AP Calculus exam.

Figure 2: Final grades of Calculus I students in research universities by experience with high school calculus. 

If we separate the grades of those who took the AP Calculus exam by their performance on this exam, we see that it makes a large difference in their final grade (Figure 3). Students who score a 1 or 2 on the AP exam are comparable to students who did not study calculus in high school, although there is a slightly lower probability of receiving a D or F or withdrawing from the course. Students with a 3 on the AB exam are comparable to the average student who took calculus in high school and did not take the AP Calculus exam, but with a somewhat higher probability of earning a B. Not surprisingly, students who earned a 4 or higher on the AB exam or a 3 or higher on the BC exam did very well in Calculus I: 45% received an A, over two thirds at least a B. It is interesting that even among these students, roughly a quarter received a D or F or withdrew from the course. In fact, the rate of DFW is remarkably consistent across all levels of preparation, suggesting that the decision to stop working or to withdraw from the course is one aspect of course performance that has little to do with high school preparation.

Figure 3: Final grades of Calculus I students in research universities by performance on AP Calculus exam.

The MAA national study of calculus, Characteristics of Successful Programs in College Calculus, is funded by NSF grant no. 0910240. The opinions expressed in this column do not necessarily reflect those of the National Science Foundation.

Measuring Teacher Quality

A perennial issue at every college and university is how to measure teacher quality. It is important because it directly influences decisions about retention, tenure, and promotion. Everyone complains about basing such decisions on end-of-course evaluations. This column will explore a recent study by Scott Carrell and James West [1], undertaken at the United States Air Force Academy (USAFA), that strongly suggests that such evaluations are even less useful than commonly believed and that the greatest long-term learning does not come from those instructors who receive the strongest evaluations at the end of the class.

The study authors chose to measure teacher effectiveness in Calculus I by examining value added in both Calculus I and Calculus II: comparing student performance on course assessments for each instructor after controlling for variables in student preparation and background that included academic background, SAT verbal and math scores, sex, and race and ethnicity. It is generally acknowledged that better teachers produce better results in their students, but this has only been extensively studied in elementary students, and even there it is not without its problems. The authors reference a 2010 study by Rothstein [2] that shows a strong positive correlation between the quality of fifth grade teachers and student performance on assessments taken in fourth grade, suggesting a significant selection bias: The best students seek out the best teachers. This may be even truer at the university level where students have much more control over who they take a class with. For this reason, Carrell and West were very careful to measure the comparability of the classes. At USAFA, everyone takes Calculus I, and there is little personal choice in which section to take, so such selection bias is less likely to occur. The authors also tested for and found no evidence of backward correlation, that the best Calculus II instructors were correlated with higher grades in Calculus I.

The authors had a large sample size with which to work, all of the students who took Calculus I from fall 2000 through spring 2007, over 10,000 students and 91 instructors. The faculty make-up at USAFA is unusual among post-secondary institutions. There is a small core of permanent faculty. Only 15% of Calculus I instructors held the rank of Associate or Full Professor, and only 31% held a doctorate in mathematics or a mathematical science. Most of the teaching is done by officers who hold a master’s degree and are doing a rotation through USAFA. The average number of years of teaching experience among all Calculus I instructors was less than four years. Because of this, there is tight control on these courses, which facilitates a careful statistical study. There are common syllabi and examinations. All instructors get to see the examinations before they are given so that there is opportunity, if an instructor so wishes, to “teach to the test,” emphasizing those parts of the curriculum that are known to be important for the assessment.

Positive responses to the following prompts all had positive influence on student performance in Calculus I, significant at the 0.05 level:

1. Instructor’s ability to provide clear, well-organized instruction.
2. Value of questions and problems raised by instructor.
3.  Instructor’s knowledge of course material.
4. The course as a whole.
5.  Amount you learned in the course.
6. The instructor’s effectiveness in facilitating my learning in the course.

Surprisingly, these evaluations of the Calculus I instructor all had negative impact on student performance in Calculus II, with responses 1, 2, and 6 significant at the 0.05 level.

On the other hand, faculty rank, highest degree, and years of teaching experience were negatively correlated with examination performance in Calculus I, but positively correlated with performance in Calculus II, with statistical significance for years of teaching experience for both the negative impact in Calculus I and the positive impact in Calculus II.

The suggested implication is that less experienced instructors tend to focus on the particular skills and abilities needed to succeed in the next assessment and that students like that approach. Experienced instructors may pay more attention to the foundational knowledge that will serve the student in subsequent courses, and students appear to be less immediately appreciative of what these instructors are able to bring to the class.

This study strongly suggests that end of course student evaluations are, at best, an incomplete measure of an instructor’s effectiveness. It also suggests a long-term weakness of simply preparing students for their next assessment, though it should be emphasized that this represents merely a guess as to why less experienced instructors appear to get better performance from their students in Calculus I assessments.

At Macalester College, we recognize the importance of student reflection on what they learned months or years earlier. When a promotion or tenure case comes to the Personnel Committee, we collect online student evaluations of that faculty member from all of the students who have taken a course with him or her over roughly the past five years, combining both recent and current appraisals with longer term assessments of the effect that instructor has had.

References:

[1] Scott E. Carrell & James E. West, 2010. "Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors," Journal of Political Economy, University of Chicago Press, vol. 118(3), pages 409-432, 06. Available at http://www.nber.org/papers/w14081

[2] Jesse Rothstein, 2010. “Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement.” Quarterly Journal of Economics 125 (1): 175–214. Available at  http://gsppi.berkeley.edu/faculty/jrothstein/published/rothstein_vam_may152009.pdf

Who Needs Algebra II?

In May this year, the National Center on Education and the Economy (NCEE) released its report, "What Does It Really Mean to Be College and Work Ready?" [1]. The report is in two parts: Mathematics and English Literacy. It is based on a national study of the proficiencies required and actually used for the most popular associate’s degree programs at two-year colleges. Just a few weeks earlier, Jordan Weissman published a piece in The Atlantic, "Here’s How Little Math Americans Actually Use at Work" [2]. That was based on a 2010 report written by Michael Handel at Northeastern University, "What Do People Do at Work? A Profile of U.S. Jobs from the Survey of Workplace Skills, Technology, and Management Practices" [3]. This large-scale survey includes an assessment of what mathematics is actually used in the workplace.

I hope that it will come as a surprise to no one that not everyone actually uses the contents of Algebra II in their work or that College Algebra taught in two-year colleges is essentially high school Algebra II. Weissman highlights Handel’s data that less than a quarter of all workers use any mathematics that is more advanced than fractions, ratios, and percentages. He raises the question whether requiring Algebra II for high school graduation is placing an unnecessary roadblock in the way of too many students. NCEE poses more nuanced questions. Why are so many students being hurried through the critical early mathematics that they will need to be work and college ready, especially fractions, ratios, and percentages, just so that they can get to Algebra II? Isn’t there a better way to prepare them for what they will need?

I will begin with the data. Handel divided the workforce into five categories:

• Upper White Collar (management, professional, technical occupations)
• Lower White Collar (clerical, sales)
•  Upper Blue Collar (craft and repair workers, construction trades, mechanics)
• Lower Blue Collar (factory workers, truck drivers)
• Service (food service, home health care, child care, janitors)

Generally, the best paying and most desirable jobs are Upper White Collar (UWC) and Upper Blue Collar (UBC). We should be equipping our students so that they can aspire to such jobs. It’s still not true that everyone needs Algebra II, but 35% of UWC workers reported using basic algebra, geometry, and/or statistics in their work. This level of mathematics is even more important for UBC workers, with 41% reporting using mathematics at the level of basic algebra, geometry, and/or statistics. This is still not Algebra II (which Handel lists as “complex Algebra” as opposed to “basic Algebra”), which was reported being used by 14% of UWC workers and 16% of UBC workers. Much less is it Calculus, which was reported by 8% of both UWC and UBC workers. But about 40% of those working in UWC or UBC jobs need a working knowledge of some high school mathematics, a higher bar than simply having passed the relevant courses. It is interesting to observe that UBC workers are more likely to use mathematics than UWC workers.

The NCEE report looked at the mathematics required for the nine most popular associate’s degree programs at two-year colleges: Accounting, Automotive Technology, Biotech/Electrical Technology, Business, Computer Programming, Criminal Justice, Early Childhood Education, Information Technology, and Nursing, as well as the General Track. This ties nicely to the Handel study because the nine are generally seen as preparation for UWC or UBC careers. NCEE selected seven two-year colleges in seven states and examined the texts, assignments, and exams in the introductory courses for these disciplines as well as for the mathematics courses required for these fields. There were three notable insights:

First, except for some work on geometric visualization, NCEE found no content in either College Algebra or Statistics, two college-credit bearing courses, that goes beyond the high school curriculum described in the Common Core State Standards in Mathematics (CCSS-M). They found that College Algebra had a large component of middle school topics, especially CCSS-M for grades 6–8 in Expressions and Equations, Functions, Number Systems, Geometry, and Ratios and Proportions. Statistics was a mix of CCSS-M middle and high school level statistics, with a significant component of grades 6–8 Ratios and Proportions and Expressions and Equations.

Second, the introductory textbooks in the disciplinary fields used nothing beyond Algebra I. Ratios and proportions are important as well as interpreting quantitative relationship expressed in tables, graphs, and formulae, but, as the report says,

When mathematics is present in the texts, equations are not solved, quadratics are absent, and functions are present but not named or analyzed, just treated as formulae. […] Students do not have to perform algebraic manipulations nor construct graphs or tables. […] The area of high school content with the highest representation in the texts, Number Systems, is found in six percent of the text chapters. [p. 16]

Third, the mathematical knowledge that was tested in these introductory courses in the disciplinary fields was far lower than what was in the textbooks. Not only was there nothing requiring Algebra II on the exams, the NCEE team could find nothing, or almost nothing, that reflected knowledge of Algebra I. Furthermore, the questions that were asked on examinations were of low difficulty. The NCEE team used the PISA (Program for International Student Assessment) Item-Difficulty Coding Framework with four levels. Examples of what is expected at each level include

• Level 0: perform simple calculations and make direct inferences;
• Level 1: use simple functional relationships and formal mathematical symbols, interpret models;
• Level 2: use multiple relationships, manipulate mathematical symbols, modify existing models; and
•  Level 3: solve multi-step application of formal procedures, evaluate arguments, create models.

The team found that over 60% of the mathematical questions on the examinations given in introductory courses were at Level 0. Few rose to Level 2, much less Level 3. (This was not the case in College Algebra and Statistics where most of the examination items were at Level 1 or 2 and some attained Level 3. This suggests that even though the material of College Algebra and Statistics does not go beyond topics covered in CCSS-M, the level of expected proficiency may be higher than what is typically encountered in high school.)

NCEE did find three mathematical topics required for the introductory courses that are not covered in CCSS-M nor in the College Algebra or Statistics classes: complex applications of measurements, schematic diagrams (2-D schematics of 3-D objects and flow charts), and geometric visualization. They also found a much greater demand for knowledge of statistics, probability, and modeling (“how to frame a real-world problem in mathematical terms”) than is commonly taught in most mainstream high school mathematics programs today.

What makes the NCEE report even more depressing is that it restricted its attention to college-credit bearing courses. Most of the mathematics taught at two-year colleges is below the level of College Algebra (see Figure 1). The mathematical requirements for UWC and UBC jobs may not be high, but we do not seem to be doing a very good job of preparing students even for what they will need.

Figure 1. Fall term Mathematics course enrollments (thousands). “Introductory” includes College Algebra, Trigonometry, and Precalculus.

Source: CBMS.

All of this raises serious questions about whether Algebra II should be expected of all graduating high school students. This parallels the situation that has been my primary concern: Should Calculus be expected of all graduating high school students who are going directly into a four-year undergraduate program, especially those who may need to take Calculus in college? I would far prefer a student who can operate at PISA Level 3 in Algebra I over a student who cannot handle problems above Level 1 in Algebra II. I would prefer Level 3 in Precalculus over Level 1 in Calculus. When students are short-changed in their mathematical preparation simply so that Algebra II or Calculus appears on the high school transcript, with little regard to what that actually means, then neither they nor society as a whole are well served.

It also raises questions about what mathematics should be required for an associate’s degree. College Algebra constitutes a significant hurdle for most two-year college students. Should there be alternatives? In this case, I believe that most two-year college students would be better served with a program that combines demanding use of the topics of Algebra I with a college-level introduction to Statistics.

We are not at the point where we can demand Algebra II for high school graduation. To do so would either create unacceptable rates of high school failure or force us to change what we mean by “understanding Algebra II.” But I worry that if we simply lower our sights and decide that, since few of our students actually will use anything from Algebra II once they have graduated, it should not be expected for graduation, then that will actually weaken the preparation that occurs in the earlier grades. Elementary and middle school mathematics should be laying the foundation for a student to succeed in Algebra II. If we want our students to have a strong working knowledge of the high school mathematics that is needed for 40% of the UWC and UBC jobs, then we want them to have the mathematical preparation that would enable them to succeed in Algebra II.

References:

[1] National Center on Education and the Economy. 2013. What does it really mean to be college and work ready? The mathematics requirements of first year community college students. Washington, DC. Available at http://www.ncee.org/college-and-work-ready/

[2] Jordan Weissman. April 24, 2013. Here’s how little Math Americans actually use at work. The Atlantic. Available at http://www.theatlantic.com/business/archive/2013/04/heres-how-little-math-americans-actually-use-at-work/275260/

[3] Michael Handel. 2010. What do people do at work? A Profile of U.S. jobs from the Survey of Workplace Skills, Technology, and Management Practices. OECD (forthcoming). Available at http://www.northeastern.edu/socant/?page_id=366

MAA Calculus Study: Graphing Calculators and CAS

This column continues my report on results of the MAA National Study of Calculus I, Characteristics of Successful Programs in College Calculus. This month I am sharing what we learned about the use of graphing calculators (with or without computer algebra systems) and computer software such as Maple or Mathematica. Our results draw on three of the surveys:

• Student survey at start of term: We asked students how calculators and/or computer algebra systems (CAS) were used in their last high school mathematics class and how comfortable they are in using these technologies.
• Student survey at end of term: We asked students how calculators or CAS had been used both in class and for out of class assignments.
• Instructor survey at start of term: We asked instructors what technologies would be allowed on examinations and which would be required on examinations.

Our first question asked students how calculators were used on exams in their last high school mathematics class (see Figure 1). As in previous columns, “research” refers to the responses of students taking Calculus I at research universities (highest degree in mathematics is doctorate), “undergrad” refers to undergraduate colleges (highest degree is bachelor’s), “masters” to masters universities (highest degree is masters), and “two-year” to two-year colleges (highest degree is associate’s).

  Figure 1. GC = graphing calculator. CAS = graphing calculator with computer algebra system capabilities (e.g. TI-89 or TI-92).

There are several interesting observations to be made from this graph. First, not surprisingly, almost all Calculus I students reported having used graphing calculators on their exams at least some of the time (“always” and “sometimes” were mutually exclusive options). Second, there is a difference by type of institution. Students at undergraduate colleges were most likely to have used graphing calculators on high school exams (94%), then those at research universities (91%), then masters universities (86%), and finally two-year colleges (77%). The differences are small but statistically significant. My best guess is that these are reflections of the economic background of these students. A second observation is that for most students, access to a graphing calculator was not always allowed. However, it is still common practice in high schools (roughly one-third of all students) to always allow students to use graphing calculators on mathematics exams.

Another striking observation from Figure 1 is that the percentage of students who were always allowed to use graphing calculators on exams is almost identical to the percentage of students who were always allowed to use graphing calculators with CAS capabilities on exams. For all categories of students, over half of them were allowed to use graphing calculators with CAS capabilities at least some of the time, which suggests that over half of the students in college Calculus I own or have had access to such calculators.

The next graph (Figure 2) shows how students at the start of the term reported their comfort level with using graphing calculators or computer algebra systems (Maple and Mathematica were provided as examples of what we meant). The most interesting feature of this graph is that students at two-year colleges are much more likely to be comfortable with Maple or Mathematica than those at four-year programs. I suspect that the reason behind this is that most Calculus I students at two-year colleges are sophomores who took pre-calculus at that college the year before. This gave them more opportunity to experience these computer algebra systems.

Figure 2. Student attitude toward use of graphing calculator or CAS on a computer such as Maple or Mathematica.

The graphs in Figures 3–5 show what students reported at the end of the term about use of technology. For the graph in Figure 3, students were asked how frequently each of these occurred in class. Percentage shows the fraction of students who responded “about half the class sessions,” “most class sessions,” or “every class session.” We note large differences in instructor use of technology generally (for this question, “technology” was not defined), and especially sharp differences for instructor use of graphing calculators or CAS (with Maple and Mathematica given as examples). It is interesting that students are most likely to encounter computer algebra systems in undergraduate and two-year colleges, much less likely in masters and research universities.

 Figure 3. End of term student reports on frequency of use of technology (at least once/month). For this question, CAS refers to a computer algebra system on a computer, such as Maple or Mathematica.

The first two sets of bars in Figure 4 show student responses to “Does your calculator find the symbolic derivative of a function?” The first set gives the percentage responding “N/A, I do not use a calculator.” The second set displays the percentage responding “yes.” Looking at the complement of these two responses, we see that across all types of institutions, roughly 50% of students taking Calculus I own a graphing calculator without CAS capabilities. The third set records the percentage responding “yes” to the question, “Were you allowed to use a graphing calculator during your exams?” Note that there are some discrepancies between what students and instructors report about allowing graphing calculators on exams (Figures 4 and 6), but the basic pattern that graphing calculators are allowed far less frequently at research universities than at other types of institutions is consistently demonstrated.

 Figure 4. End of term student reports on calculator use. No calculator = do not use a calculator. Calculator with CAS = use a calculator with CAS capabilities. Calc allowed on exams = graphing calculators were allowed on exams.

We also asked how often “The assignments completed outside of class time required that I use technology to understand ideas.” Again, we see much less use of technology at research universities, the greatest use at undergraduate and two-year colleges.

 Figure 5. Frequency with which technology (either graphing calculators or computers) was used for out of class assignments. Almost never = less than once per month (includes never). Sometimes = at least once per month but less than once per week. Often = at least once per week.

The last two graphs (Figures 6 and 7) are taken from the instructor responses at the start of the term: what technology they would allow on their exams and what technology they would require on their exams. Again, we see a clear indication that technology, especially the use of graphing calculators without CAS capabilities, is much less common at research universities than other types of institutions.

It is interesting to observe that there are large numbers of instructors who allow but do not require technology on the exams. At research universities, 26% require the use of some kind of technology, and a further 25% allow but do not require the use of some sort of technology. For undergraduate colleges, 38% of instructors require technology, an additional 42% allow it. At masters universities, 42% require, and a further 33% allow. At two-year colleges, 52% require, and an additional 36% allow.

 Figure 6. Start of term report by instructor of intended use of technology on exams. GC = graphing calculator. Most of those who checked “other” reported that they allowed graphing calculators on some but not all parts of the exam. Some reported allowing only scientific calculators.

Figure 7. Start of term report by instructor of intended use of technology on exams. GC = graphing calculator. Most of those who checked “other” reported that they required graphing calculators on some but not all parts of the exam. Some reported requiring only scientific calculators.

We see a pattern of very heavy use of graphing calculators in high schools, driven, no doubt, by the fact that students are expected to use them for certain sections of the Advanced Placement Calculus exams. They are still the dominant technology at colleges and universities, but there the use is as likely to be voluntary as required. This implies that in many colleges and universities questions and assignments are posed in such a way that graphing calculators confer little or no advantage. The use of graphing calculators at the post-secondary level varies tremendously by type of institution. Yet even at the research universities, over half the instructors allow the use of graphing calculators for at least some portions of their exams. 

The MAA national study of calculus, Characteristics of Successful Programs in College Calculus, is funded by NSF grant no. 0910240. The opinions expressed in this column do not necessarily reflect those of the National Science Foundation.

MAA Calculus Study: Progressive Teaching

Last month (MAA Calculus Study: Good Teaching) I discussed the student-described attributes of instructors that were highly correlated with improvements in student confidence, enjoyment of mathematics, and desire to continue to study mathematics. This month I will discuss a second set of instructor attributes that we are labeling "Progressive Teaching" because they are generally associated with approaches to teaching and learning that focus on active engagement of the students.

Here the evidence for improved results is less clear. In particular, Sadler and Sonnert discovered a strong interaction with the attributes we are calling "Good Teaching": teachers who rated high on Good Teaching improved student outcomes if they also rated high on Progressive Teaching. But if they rated low on Good Teaching, then a high rating on Progressive Teaching had a strongly negative effect on student confidence. This might have been expected. Good Teaching describes student-teacher interactions, including the degree to which students feel encouraged to participate in class and supported by the instructor. It is not surprising that students who are encountering unfamiliar approaches to classroom learning react negatively if they believe that that the instructor is not encouraging or supportive.

We also have evidence of some consistently positive effects from Progressive Teaching. Even with a low score on Good Teaching, Progressive Teaching was seen to be helpful in convincing students to continue the study of mathematics. Our conclusions are that:

a.       Good Teaching and Progressive Teaching are independent clusters of student perceptions of instructor behaviors,

b.      Good Teaching is more important to student persistence than Progressive Teaching,

c.       both can serve to improve student outcomes, and

d.      teaching is most effective when instructors rate high on both measures.

There were 12 student responses that clustered into what we are calling Progressive Teaching:

My calculus instructor frequently
1.      Assigned sections of the textbook to read before coming to class.

2.      Had students work with one another.
3.      Had students give presentations.
4.      Asked students to explain their thinking in class.
5.      Required students to explain their thinking on homework assignments.
6.      Required students to explain their thinking on exams.
7.      Held whole class discussions.

My calculus instructor did not frequently
8.      Lecture.

Assignments completed outside of class
9.      Required that I solve word problems.
10.  Were problems unlike those done in class or in the book.
11.  Were often submitted as a group project.
12.  Were returned with helpful feedback and comments.

With one exception, the following graphs show the percentage of students who reported that their instructors employed each of these practices often or very often (a 5 or 6 on a Likert scale from 1 = not at all to 6 = very often). The exception is practice #8. Here we record the percentage of students who responded 1, 2, or 3 on the same scale to the question, "During class time, how frequently did your instructor lecture?".

We see that for most of the instructor behaviors (practices 1 through 8), the undergraduate colleges and two-year colleges are where these are most likely to be employed. The relatively large percentage of instructors at masters universities who had students give presentations in class (13% as opposed to 6% at all other types of institutions) is still small and may be an artifact of the relatively small number of responses from students at masters universities (305 students at 18 institutions).  The research universities are where we find the most challenging problems being posed on assignments, either word problems or those unlike those done in class or in the book. Instructors at two-year colleges provide the most helpful feedback on assignments, instructors at research universities the least helpful feedback.

Figure 1: Instructor practices 1 through 3 and 8

Figure 2: Instructor practices 4 through 7

Figure 3: Instructor practices 9 through 12

The MAA national study of calculus, Characteristics of Successful Programs in College Calculus, is funded by NSF grant no. 0910240. The opinions expressed in this column do not necessarily reflect those of the National Science Foundation.

MAA Calculus Study: Good Teaching

One of the primary goals of the MAA Calculus Study, Characteristics of Successful Programs in College Calculus (NSF #0910240), has been to identify the factors that are highly correlated with an improvement in student attitudes from the start to the end of the calculus course: confidence in mathematical ability, enjoyment of mathematics, and desire to continue the study of mathematics. To this end, Phil Sadler and Gerhard Sonnert of the Science Education Department within the Harvard-Smithsonian Center for Astrophysics constructed a hierarchical linear model from our survey responses to identify these factors. The factors reside at three levels: institutional, classroom, and individual student. Not surprisingly, most of the variation in student attitudes can be explained by student background, but there are influences at the institutional and classroom level. We have been particularly interested in what happens at the classroom level where there is the greatest opportunity for improvement.

Sadler and Sonnert ran a factor analysis of the classroom-level variables, clumping those responses that were highly correlated. They discovered that the responses broke into three distinct clusters, which we are labeling “technology,” “progressive teaching,” and “good teaching” because these seem to describe the characteristics of the instruction. By far, the most important of these in terms of high correlation with improved attitudes is “good teaching.” Listed below are the 21 student-reported characteristics of instruction that are highly correlated with each other and highly correlated with improvements in student attitudes, characteristics that collectively we are calling “good teaching”:

My calculus instructor:

1. Asked questions to determine if I understood what was being discussed.
2. Listened carefully to my questions and comments.
3. Discussed applications of calculus.
4. Allowed time for me to understand difficult ideas.
5. Helped me become a better problem solver.
6. Encouraged students to enroll in Calculus II.
7. Acted as if I was capable of understanding the key ideas of calculus.
8. Made me feel comfortable asking questions during class.
9. Encouraged students to seek help during office hours.
10. Presented more than one method for solving problems.
11. Made class interesting.
12. Provided explanations that were understandable.
13. Was available to make appointments outside of office hours, if needed.

My calculus instructor did not:

14. Discourage me from wanting to continue taking calculus.
15. Make students feel nervous during class.

My instructor often or very often:

16. Showed how to work specific problems.
17. Asked questions.
18. Prepared extra material to help students understand calculus concepts or procedures.

In addition:

19. My calculus exams were a good assessment of what I learned.
20. My exams were fairly graded.
21. My homework was fairly graded.

The good news is that most calculus instructors rated highly on most of these characteristics. This good news needs to be tempered by two facts: Instructors could and in many cases did elect not to participate even though other instructors at their institution were involved in the study, and these responses were all collected at the end of the term. They reflect the opinions of the students who had successfully navigated this course, predominantly students who were earning an A or a B in the course (roughly 40% A, 40% B, 20% C).

It is interesting and informative to see how students at different types of institutions rated their instructors on these criteria. We followed CBMS in categorizing post-secondary institutions by the highest mathematics degree offered at that institution. I am using “research” to designate universities that offer a PhD in Mathematics (predominantly large state flagship universities), “masters” if the highest degree is a master’s (predominantly public comprehensive universities), “undergrad” if it is a bachelor’s degree (predominantly private liberal arts colleges), and “two-year” if it is an associate’s degree (predominantly community and technical colleges). As shown in the graphs at the end of this article, instructors at research universities got the lowest ratings on every characteristic except “showed how to work specific problems.” For most of these characteristics, instructors at undergraduate colleges were the next lowest, then masters universities, and most of the time instructors at two-year colleges received the highest ratings. 

There were a few notable exceptions. Instructors at undergraduate colleges received the highest ratings in some of the areas where one would expect them to be strong:

• Acted as if I was capable of understanding the key ideas of calculus.
• Encouraged students to seek help during office hours.
• Was available to make appointments outside of office hours, if needed.
• Did not make students feel nervous during class.

Masters universities scored highest in often or very often showing how to work specific problems, and just barely edged out two-year colleges in “listened carefully” and “my exams were fairly graded.”

There are a number of possible explanations for the weaknesses of research universities and the strengths of two-year colleges. One is class size. The largest classes are found at the research universities where average class size is 53, the smallest at two-year colleges where the average is 21. However, average class size at masters universities is larger than at undergraduate colleges, so class size cannot be the only explanatory variable. Some of the discrepancies between institution types may be explained by student expectations. This is because SAT scores and high school mathematics GPA are highest for research universities, then undergraduate colleges, then masters universities, and lowest for two-year colleges. Better students may have higher expectations of their instructors, or they may be more discouraged by encountering difficulties in this course. The differences may also have something to do with age and thus maturity of the students. The youngest students are at research universities, the oldest at two-year colleges. They also may be related to the relatively large number of instructors at research universities who teach calculus but have little or no interest in teaching this course, as opposed to two-year colleges where the interest is very high (see my November column, MAA Calculus Study: The Instructors). Nevertheless, it is discouraging that students at research universities seem to be getting calculus instruction that has a worse effect on student attitudes than instruction at other types of institutions.

Figure 1: Instructor Characteristics 1–5.

Figure 2: Instructor Characteristics 6–10.

Figure 3: Instructor Characteristics 11–15.

Figure 4: Instructor Characteristics 16–18.

Figure 5: Instructor Characteristics 19–21.

The MAA national study of calculus, Characteristics of Successful Programs in College Calculus, is funded by NSF grant no. 0910240. The opinions expressed in this column do not necessarily reflect those of the National Science Foundation.

Mathematics in 2025

The National Research Council of the National Academies has just released the preliminary version of its report, The Mathematical Sciences in 2025 [1]. This was produced in response to a request from the National Science Foundation. It comes as the latest in a series of glimpses into the future of mathematics that go back to the “David reports” of 1984 and 1990 [2,3] and the “Odom study” of 1998 [4]. This report is important because it will influence the direction NSF takes as it plans for the future.

The emphasis of the report is on the central role that the mathematical sciences are taking within research in areas as diverse as biology, finance, and climate science. Traditional disciplinary boundaries are blurring. There is an increasing need for scientists who are well grounded in mathematical sciences, especially the statistical and computational sciences, as well as other disciplines. This goes two ways. It means opening courses and programs in the mathematical sciences, especially at the graduate level, to those in other fields of study, and it means ensuring that students graduating in the mathematical sciences are prepared to work in this interdisciplinary world.

This has implications right down the line of mathematics education. The authors of the report question whether, in a scientific world that is dominated by big data and the challenges of large-scale computation, the traditional calculus-focused curriculum is the most appropriate for all students. As they say, “Different pathways are needed for students who may go on to work in bioinformatics, ecology, medicine, computing, and so on. It is not enough to rearrange existing courses to create alternative curricula; a redesigned offering of courses and majors is needed [my emphasis].” (NRC 2013, p. S-9)

The report also stresses the importance of attracting more women and students from traditionally underrepresented minorities to the mathematical societies. This is the one place where I disagree with the report, for it asserts that, “While there has been progress in the last 10–20 years, the fraction of women and minorities in the mathematical sciences drops with each step up the career ladder.” (NRC 2013, p. S-10). I don’t question the drop. I question whether there has been progress over the last 10–20 years.

If we look at mathematics majors (bachelor’s degrees) by gender, we see that over the period 1990 to 2011 the number of men majoring in mathematics grew by 25% while the number of women grew by only 10% (Figure 1). As a result, the percentage of bachelor’s degrees in the mathematical sciences going to women has dropped to 43.1%, the first time it has been this low since 1981. This is having knock-on effects for graduate programs. The percentage of bachelor’s degrees in mathematics that went to women peaked in 1999 at 47.8%. The percentage of master’s degrees in mathematics that went to women peaked in 2004 at 45.1% and has since dropped back to 40.9%. The percentage of doctoral degrees in mathematics that went to women peaked in 2008 and ’09 at 31.0%. It has since dropped back to 28.6%. The good news is that the past decade has seen strong growth in the number of mathematics majors, but two-thirds of the growth since 2001 has been in the number of men.

We see an even more discouraging pattern among Black students (Figure 2). The number of Black mathematics majors is essentially back to where it was twenty years ago despite the number of bachelor’s degrees earned by Black students almost tripling over this period. The number of Black mathematics majors peaked in 1997 at 1,089. It was back down to only 840 in 2011. The number of ethnically Asian mathematics majors has been growing strongly over the past decade. Even so, the number earning undergraduate degrees in the mathematical sciences has only doubled since 1990, while the number earning bachelor’s degrees has tripled. The growth in the number of Hispanic mathematics majors looks good, having slightly more than tripled in twenty years, until you realize that the number of Hispanic students graduating from college is almost five times what it was in 1990 (154,000 versus 33,000). Where we do see strong growth, especially since 2007, is in the number of non-resident aliens majoring in mathematics, which now stands at 7% of all US mathematics majors.

I must emphasize that the NRC report does highlight the importance of increasing the participation of women and members of underrepresented groups. It includes the following specific recommendation:

Recommendation 5-4: Every academic department in the mathematical sciences should explicitly incorporate recruitment and retention of women and underrepresented groups into the responsibilities of the faculty members in charge of the undergraduate program, graduate program, and faculty hiring and promotion.  Resources need to be provided to enable departments to adopt, monitor and adapt successful recruiting and mentoring programs that have been pioneered at other schools and to find and correct any disincentives that may exist in the department. (NRC 2013, p. 5-18)

I have only touched on a few of the topics covered in the NRC report. It also discusses the increasingly important role of the mathematical sciences institutes, the issue of maintaining online repositories of mathematical research such as arXive, and the threats to mathematics departments as more instruction—especially for the service courses that often provide the justification for a large mathematics faculty—is moved online. This is a report well worth reading and pondering. 

---

[1] National Research Council. 2013. The MathematicalSciences in 2025. Washington, DC. The National Academies Press.

[2] NRC. 1984, Renewing US Mathematics: Critical Resource for the Future. Washington, DC. The National Academies Press.

[3] NRC. 1990. Renewing US Mathematics: A Plan for the 1990s. Washington, DC. The National Academies Press.

[4] NSF. 1998. Report of the Senior Assessment Panel for the International Assessment of the US Mathematical Sciences. Arlington, VA. National Science Foundation.