Strategies for Change

One of the most striking findings from the MAA’s survey of university mathematics departments undertaken this past spring (see last month’s column) is the almost universal recognition that current practice in the precalculus through single variable calculus sequence needs to be improved. Many such efforts are now underway, but many of them lack understanding of how institutional change occurs as well as recognition of the importance of this understanding.

Much of the literature on institutional change lies too far from the contexts or concerns of mathematics departments to be easily translatable, but an important paper appeared a little over a year ago in the Journal of Engineering Education that provides an insightful framework for understanding change in the context of undergraduate STEM education: “Increasing the Use of Evidence-Based Teaching in STEM Higher Education: A Comparison of Eight Change Strategies” by Borrego and Henderson (2014). This paper takes the framework distilled by Henderson, Beach, and Finkelstein in 2010 and 2011 from their literature review of change strategies and applies it to eight different approaches to bringing evidence-based teaching into the undergraduate STEM classroom. This short column cannot do justice to their extensive discussion, but it can perhaps whet interest in reading their paper.

Henderson, Beach, and Finkelstein have identified two axes along which change strategies occur (Table 1): those whose focus is on changing individuals versus those that focus on changing environments and structures, and those that they describe as prescribed, meaning that they try to implement specific solutions, versus those they describe as emergent, meaning that they attempt to foster conditions that support local actors in finding their own solutions. This results in the four categories shown in Table 1.

Table I: Change theories mapped to the four categories of change strategies. The italicized text lists two specific change strategies for each of the four categories. Reproduced from Borrego and Hnderson (2014).

Within each of the four categories, they identify two strategies that have been used. For example, under a prescribed outcome focused on individuals, Category I, they identify Diffusion and Implementation as two change strategies. Diffusion describes the common practice of developing an innovation at a single location and then publicizing it in the hope that others will pick it up. Implementation involves the development of a curriculum or specified set of practices that are intended to be implemented at other institutions. For each of the eight change strategies, they describe the underlying logic of how it could effect change, describe what it looks like in practice, and give an example of how it has been used, accompanied by some assessment of its potential strengths and weaknesses. Diffusion, in particular, is very common and is known to be capable of raising awareness of what can be done, but it often runs into challenges of incompatibility together with a lack of support for those who would attempt to implement it.

At the opposite corner are the emergent strategies that focus on environments and structures. Here Borrego and Henderson consider Learning Organizations and Complexity Leadership Theory. Learning organizations have emerged from management theory as a means of facilitating improvements. They involve informal communities of practice that share their insights into what is and is not working, embedded within a formal structure that facilitates the implementation of the best ideas that emerge from these communities. In management-speak, it is the middle-line managers who are the key to the success of this approach. In the context of higher education, these middle-line managers are the department chairs and the senior, most highly respected faculty.

The effectiveness of Learning Organizations resonates with what I have seen of effective departments. They require an upper administration that recognizes there are problems in undergraduate mathematics education and are willing to invest resources in practical and cost- effective means of improving this education, together with faculty in the trenches who are passionate about finding ways of improving the teaching and learning that takes place at their institution. The faculty need to be encouraged to form such communities of practice, sharing their understanding and envisioning what changes would improve teaching and learning. Some of the best undergraduate teaching we have seen has been built on the practice of regular meetings of the instructors for a particular class. The role of the chair and senior faculty is one of encouraging the generation of these ideas, providing feedback and guidance in refining them, and then selling the result to the upper administration, conscious of how it fits into the concerns and priorities of deans and provosts. Throughout this process, it is critical to have access to robust and timely data on student performance for this class as well as for the downstream courses both within and beyond the mathematics department.

Complexity Leadership Theory is based on recognition of the difficulties inherent in trying to change any complex institution and calls on the leadership to do three things: to disrupt existing patterns, to encourage novelty, and to make sense of the responses that emerge. Borrego and Henderson could not find any examples of Complexity Leadership Theory within higher education, but, as I interpret this approach as it might appear within a mathematics department, it speaks to the responsibility of the chair and leading faculty to draw attention to what is not working, to encourage faculty to seek creative solutions to these problems, and then to shape what emerges in a way that can be implemented. In many respects, it is not so different from Learning Organizations. The strategies of Category IV highlight the key role of the departmental leadership, which must involve more than just the chair or head of the department.

In their discussion, Borrego and Henderson emphasize that they are not suggesting a preference for any of these categories, although they do note that Category I is the most common within higher education and Category IV the least. My own experience suggests that the strategies of Category IV have the greatest chance of making a lasting improvement. Nevertheless, anyone seeking systemic change will need to employ a variety of strategies that span all of these approaches. Their point is that anyone seeking change must be aware of the nature of what they seek to accomplish and must recognize which strategies are best suited to their desired goals.

Bibliography

M. Borrego and C. Henderson. 2014. Increasing the use of evidence-based teaching in STEM higher education: A comparison of eight change strategies. Journal of Engineering Education. 103 (2): 220–252.

C. Henderson, A. Beach, N. Finkelstein. 2011. Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching. 48 (8): 952–984.

C. Henderson, N. Finkelstein, A. Beach. 2010. Beyond dissemination in college science teaching: An introduction to four core change strategies. Journal of College Science Teaching. 39 (5): 18–25.

MAA Calculus Study: A New Initiative

With the publication of Insights and Recommendations from the MAA National Study of College Calculus, we are wrapping up the original MAA calculus study, Characteristics of Successful Programs in College Calculus (CSPCC, NSF #0910240). This past January, MAA began a new large-scale program, Progress through Calculus (PtC, NSF #1430540), that is designed to build on the lessons of CSPCC. I am continuing as PI of the new project. Co-PIs Chris Rasmussen at San Diego State, Sean Larsen at Portland State, Jess Ellis at Colorado State, and senior researcher Estrella Johnson at Virginia Tech are leading local teams of post-docs, graduate students, and undergraduates who will be working on this effort.

CSPCC sought to identify what made certain calculus programs more successful than others but was limited in its measures of success to what could be learned about changes in student attitudes between the start and end of Calculus I and to what could be observed from a single three-day visit to a select group of 20 colleges and universities. PtC is extending its purview to the entire sequence of precalculus through single variable calculus, and it will take broader measures of success, including performance on a standardized assessment instrument, persistence into subsequent mathematics courses, and performance in subsequent courses. It also is shifting emphasis from description of the attributes of successful programs to analysis of the process of change: What obstacles do departments encounter as they attempt to improve the success of their students? What accounts for the difference between departments that are successful in institutionalizing improvements and those that are not?

We began this past spring with a survey of all mathematics departments offering a graduate degree in Mathematics, either MA/MS or PhD. This is a manageable number of institutions: 178 PhD and 152 Masters universities. These are the places that most often struggle with large classes and with the trade-off between teaching and research. We had an excellent participation rate: 75% of PhD and 59% of Masters universities filled out the survey.

Data from this survey will appear in future papers and articles, but for this column I want to focus on the most important information we learned: what these departments see as critical to offering successful classes and how that compares to how well they consider themselves to be doing on these measures.

CSPCC identified eight practices of successful programs. These are listed here in the order implied by the number of doctoral departments in the PtC survey that identified each as “very important to a successful precalculus/calculus sequence.”

1. Student placement into the appropriate initial course 
2. GTA teaching preparation and development 
3. Student support programs (e.g. tutoring center) 
4. Uniform course components (e.g. textbook, schedule, homework) 
5. Courses that challenge students 
6. Active learning strategies 
7. Monitoring of the precalculus/calculus sequence through the collection of local data 
8. Regular instructor meetings about course delivery.

The graphs in Figures 1 and 2 show the percentage of respondents who identified each as “very important” (as opposed to “somewhat important” or “not important”), as well as the percentage of respondents who considered themselves to be “very successful” with each (opposed to “somewhat successful” or “not successful”).

Figure 1. PhD universities. What they consider to be important versus how successful they consider themselves to be.

Figure 2. Masters universities. What they consider to be important versus how successful they consider themselves to be.

What is most interesting for our purposes is where departments see a substantial gap between what they consider to be very important and where they see themselves as very successful. These are the areas where departments are going to be most receptive to change. If we look for large absolute or relative gaps, five of the eight practices show up as areas of concern (Table 1). The biggest absolute gap is for placement; approximately half of all universities consider placement to be very important but do not rate themselves as very successful. The largest relative gap is for active learning, where only 27% of doctoral universities and 36% of masters universities that consider this to be very important also consider themselves to be very successful at it.

Table 1. Departments that consider themselves to be very successful as percentage of those that consider the practice to be very important.

The next stage of this project will be the building of networks of universities with common concerns and the identification of twelve universities for intense study over a three-year period. This stage has begun with a small workshop for representatives of 27 universities, a workshop that will begin building these networks and is ending as this column goes live on November 1. It will be continuing with a larger conference in Saint Paul, MN, June 16–19, 2016. Watch this space for more information about that conference.

Evidence for IBL

Special Note: The AMS Blog On Teaching and Learning Mathematics has started a six-part series on active learning.

Over the past decade, the Educational Advancement Foundation has supported programs to promote Inquiry-Based Learning (IBL) in mathematics at four major universities. IBL is not a curriculum. Rather, it is a guiding philosophy for instruction that takes a structured approach to active learning, directing student activities and projects toward building a fluent and comprehensive understanding of the central concepts of the course. Ethnography & Evaluation Research (E&ER) at the University of Colorado, Boulder has studied the effectiveness of these implementations. Several research papers have resulted, of which the paper by Kogan and Laursen (2014), discussed in my column Evidence of Improved Teaching (October 2013), presented very clear evidence that IBL prepares students for subsequent courses better than standard instruction and that IBL can result in students taking more mathematics courses, especially when offered early enough in the curriculum. Two recent papers document the benefits of IBL in preparing future teachers and in building personal empowerment.

In Implementation and outcomes of inquiry-based learning in mathematics content courses for pre-service teachers (Laursen, Hassi, and Hough, 2015), the authors focused on the development of Mathematical Knowledge for Teaching (MKT), a term coined by Deborah Ball to describe the kind of knowledge that teachers must draw upon to teach mathematics well and that reflects understanding of how ideas and concepts relate to one another as well as the common difficulties and misunderstandings that students are likely to encounter. Being prepared for teaching requires more than being able to find solutions to particular problems. A good teacher must have at her or his disposal a variety of approaches to a solution and the ability to take a student’s incorrect attempt at an answer, recognize where the misunderstanding lies, and build on what the student does understand.

In theory, IBL should help develop MKT because it focuses on precisely those characteristics of practicing mathematicians that teachers most need, the habits of mind than include sense-making, conjecture, experimentation, creation, and communication.

E&ER studied students in thirteen sections of seven courses for pre-service teachers at two of the four universities, courses that collectively spanned preparation for primary, middle school, and secondary teaching. They used an instrument developed by Ball and colleagues, Learning Mathematics for Teaching (LMT) that has been validated as an effective measure of MKT for practicing teachers. The results were impressive. The students had begun the term with LMT scores that averaged at the mean for in-service teachers across the country. Each of the IBL classes saw mean LMT scores rise by 0.67 to 0.90 standard deviations. In line with the results of the 2014 report, all students experienced gains from IBL, but the weakest students saw the greatest gains.

The second recent article is Transforming learning: Personal empowerment in learning mathematics (Hassi and Laursen, 2015). In Adding It Up (NRC 2001), mathematical proficiency is recognized as consisting of five strands: conceptual understanding, procedural fluency, strategic competence, adaptive reasoning, and productive disposition. They are each critically important. This paper investigates the effect of IBL on both strategic competence, what the authors term cognitive empowerment, and productive disposition, which they separate into self-empowerment and social empowerment, the last of which also incorporates effective communication.

The study was conducted through interviews with students who had taken a class at one of the four universities using IBL. An overwhelming majority of students reported gains in each of the three areas of personal empowerment. Among women 77% and among men 69% reported an increase in self-esteem, sense of self-efficacy, and confidence from their IBL experience. For general thinking skills, deep thinking and learning, flexibility, and creativity, 77% of the women and 90% of the men described improvements. For ability to explain and discuss mathematics as well as skills in writing and presenting mathematics, 79% of the women and 76% of the men saw gains.

When pressed for what made the IBL experience special, students identified their own role in influencing the course pace and direction, the importance of combining both individual and collaborative work, and the fact that they were faced with problems that were both challenging and meaningful. They appreciated that they were given responsibility to think on their own. Such experiences were especially important for women and for first-year students.

In the very discouraging reports on the effects of Calculus I instruction in most US universities (Sonnert and Sadler 2015), we see courses that accomplish exactly the opposite of personal empowerment, courses that sharply decrease student confidence and sense of self-efficacy. It does not have to be this way.

References

Hassi, M.-L., and Laursen, S.L. 2015. Transformative learning: Personal empowerment in learning mathematics. Journal of Transformative Education. Published online before print May 24, 2015, doi: 10.1177/1541344615587111.

M. Kogan and S. Laursen. 2014. Assessing long-term effects of inquiry-based learning: A case study from college mathematics. Innovative Higher Education 39 (3), 183–199. http://link.springer.com/article/10.1007/s10755-013-9269-9

Laursen, S.L., Hassi, M.-L., and Hough, S. 2015. Implementation and outcomes of inquiry-based learning in mathematics content courses for pre-service teachers. International Journal of Mathematical Education in Science and Technology. Published online before print July 25, 2015, doi: 0.1080/0020739X.2015.1068390

National Research Council (NRC). 2001 Adding it up: Helping children learn mathematics. J. Kilpatrick, J. Swafford, and B. Findell (Eds.). Washington, DC: National Academy Press.

Sonnert, G. and Sadler, P. 2015. The impact of instructor and institutional factors on students’ attitudes. Pages 17–29 in Insights and Recommendations from the MAA National Study of College Calculus, D. Bressoud, V. Mesa, and C. Rasmussen (Eds.). Washington, DC: Mathematical Association of America Press.

Calculus at Crisis V: Networks of Support

Special Notice: The MAA Notes volume summarizing the results of Characteristics of Successful programs in College Calculus (NSF #0910240), Insights and Recommendations from the MAA National Study of College Calculus, is now available for free download as a PDF file at www.maa.org/cspcc.

This is the last of my columns on Calculus at Crisis. In the first three, from May, June, and July, I explained why we can no longer afford to continue doing what we have always done. Last month I described some of the lessons that have been learned in recent years about best practices with regard to placement, student support, curriculum, and pedagogy. Unfortunately, as those who seek to improve the teaching and learning of introductory mathematics and science have come to realize, knowing what works is not enough.

There are many barriers to change, both individual and institutional. Lack of awareness of what can be done is seldom one of them. In recent years, leaders in physics and chemistry education research, especially Melissa Dancy, Noah Finkelstein, and Charles Henderson have studied these barriers and begun to translate insights from the study of how institutional change comes about in order to assist those who seek to improve post- secondary science, mathematics, and engineering education.

One of the best short summaries describing specific steps toward achieving long-term change is Achieving Systemic Change, a report issued jointly by the American Association for the Advancement of Science (AAAS), the American Association of Colleges and Universities (AAC&U), the Association of American Universities (AAU), and the Association of Public and Land-grant Universities (APLU) that I discussed this past December. Its emphasis on creating supportive networks within and across institutions is reflected in our own findings in the MAA’s calculus study.

There has always been lively interest from individual faculty members in improving mathematics education. Heroic efforts have often succeeded in moving the dial, but without strong departmental support they are not sustainable. As I have explained over the past months, deans, provosts, and even presidents now realize that something must be done. I have yet to meet a dean of science who is not willing—usually even eager—to fund a proposal from the mathematics department for improving student outcomes provided it is concrete, workable, and cost-effective. (Just hiring more mathematicians does not cut it.) The key link between eager faculty and concerned administrators is the department chair, together with the senior, most highly respected faculty. Without their support and cooperation, no lasting improvements are possible.

The department chair is essential. This is the person who can take an enthusiastic proposal and massage it into a workable plan whose benefits are understandable to the upper administration. This is the person who can take a request from the dean, understand the resources that will be required, and find the right people to work on it. Unfortunately, appointment as chair does not automatically confer such wisdom. Part of what is needed is an understanding of what is being done at comparable institutions, how it is being implemented, what is working or failing and why. This is where the mathematical societies have an important role to play. AMS does this through its Information for Department Leaders, the work of the Committee on Education, and its blog On the Teaching and Learning of Mathematics. The MAA’s CUPM, CTUM, and CRAFTY committees provide this information through publications, panels, and contributed paper sessions. SIAM, ASA, and AMATYC also embrace this mission. Common Vision began this year as an effort to coordinate these activities across the five societies.

But a supportive department chair is not enough. The lasting power center in any department consists of senior faculty who are highly respected for their research visibility. The most successful calculus programs we have seen in the MAA study Characteristics of Successful Programs of College Calculus involved some of these senior faculty in an advisory capacity: monitoring the annual data on student performance, observing occasional classes, mentoring graduate students not just for research but also for the development of teaching expertise, and providing encouragement and a sounding board to those—usually younger faculty—engaged in trying new methods in the classroom. It will be the chair’s responsibility to identify the right people for this advisory group, but once it is in existence it can help ensure that future chairs are sympathetic to these efforts.

Finally, any mathematics department seeking to improve undergraduate education must remember that it is not alone within its institution. Similar efforts are underway in each of the sciences as well as engineering. Deans and provosts can help by formally recognizing those who serve in these senior roles across all STEM departments and encouraging links between these groups of faculty. They can draw on support and advice from consortia of colleges and universities such as AAU, APLU, and AAC&U, as well as multidisciplinary societies and consortia such as AAAS and the Partnership for Undergraduate Life Science Education (PULSE), all of whom are working to promote networks of educational innovation that cross STEM disciplines. Joining with other departments within the institution can dispel the perception of mathematics as insular and unconcerned with the needs of others as it strengthens individual departmental efforts. All STEM departments are facing similar difficulties. This crisis presents us with an exceptional opportunity to work across traditional boundaries.

Calculus at Crisis IV: Best Practices

In my last three columns I explained the reasons that college calculus instruction is now at crisis:

1. The need to teach ever more students, who often bring weaker preparation, using fewer resources.
2. The fact that most Calculus I students have already studied calculus in high school (this past spring 424,000 students took an AP Calculus exam, an increase of 100,000 over the past five years).
3. The pressures from the client disciplines to equip their students with the mathematical knowledge and habits of mind that they actually will need.

As I have traveled this country to meet with mathematics departments, I have seen that there is a general recognition on the part of chairs, deans, provosts, and occasionally even presidents that the past solutions for calculus instruction are no longer adequate. I am encouraged by the fact that the mainstream calculus sequence is so central to all of the STEM disciplines that, even in these tight budget times, many deans and provosts can find the resources to support innovative programs if they can be convinced these efforts are sustainable, cost-effective, and will actually make a difference.

There are four basic leverage points for improving the calculus sequence so that it better meets at least some of these pressures: placement, student support, curriculum, and pedagogy. We know a lot about what does work for each of these. Much of this knowledge—relevant to the teaching of calculus—is contained in the new MAA publication Insights and Recommendations from the MAA National Study of College Calculus, the report on a five-year study of Characteristics of Successful Programs in College Calculus undertaken by the MAA with support from NSF (#0910240). I briefly summarize some of the insights.

Placement. Placement can have a huge impact on student success rates. However, given the demands of the client disciplines and the fact that remediation is usually of doubtful value (see The Pitfalls of Precalculus), just tightening up the requirements for access to calculus is unlikely to make a dean or provost happy. We do have evidence of the effectiveness of adaptive online exams such as ALEKS that probe student understanding to reveal individual strengths and weaknesses, especially when combined with tools that can help students address specific topics on which they need refreshing. But there is no one placement exam or means of implementation that will work for all institutions. Further elaboration on what we have learned about placement exams can be found in Chapter 5, Placement and Student Performance in Calculus I, of Insights and Recommendations.

Student Support. Programs modeled on the Emerging Scholars Programs can be very effective for supporting at-risk students (see Hsu, Murphy, Treisman, 2008). Tutoring centers are virtually universal, but not always as useful as they could be. The best we have seen put thought into the training of the tutors, require classroom instructors to hold some of their office hours in the center, and are located conveniently with a congenial atmosphere that encourages students to drop in to study or work on group projects even if they do not need the assistance of a tutor. In addition, quick identification and effective guidance of students who are struggling with the course is essential. More on these points can be found in Chapter 6, Academic and Social Supports, of Insights and Recommendations.

Curriculum. This is the toughest place at which to apply leverage. Most faculty are fine with changes to placement procedures and support services but are appalled at the very thought of touching the curriculum. The pushback against the Calculus Reform movement of the early 1990s was strongest where curricular changes were suggested. Yet this is where we are most likely to be successful in meeting the needs of students who studied calculus in high school, and it must be part of any strategy for meeting the needs of the client disciplines. Research coming out of Arizona State University and other centers of research in undergraduate mathematics education has revealed the basic wisdom of many of the Calculus Reform curricula that approached calculus as a study of dynamical systems. Curricular materials are now being developed that have a much firmer basis in an understanding of student difficulties with the concepts of calculus (for an example, see Beyond the Limit).

Pedagogy. Another aspect of the Calculus Reform movement that was poorly received was the emphasis on active learning. The evidence is now overwhelming that active learning is critical, especially important for at-risk students and essential for meeting the needs of the client disciplines. We have learned a lot in the intervening quarter century about how to do it well and cost-effectively, and this is one of the places where new technologies can be particularly helpful. There are now many models for implementation of active learning strategies, spanning classrooms of all sizes, student audiences at varied levels of expertise, and faculty with different levels of commitment to changing how they teach (see Reaching Students). Evidence for the effectiveness of active learning and recommendations of strategies for implementing it can be found in Donovan & Bransford, 2005; Freeman et al., 2014; Fry, 2014; Kober, 2015; and Kogan & Laursen, 2014.

The bottom line is that we do have knowledge that can help us face this crisis. There is no universal solution. Each department will have to find its own way toward its own solutions. But it need not stumble alone. As I will explain next month in the fifth and final column in this series, making meaningful and lasting change requires networks of support both within and beyond the individual department. Here also our knowledge base of what works and why has expanded in recent years.

References

Bressoud, D., Mesa, V., Rasmussen, C. (eds.) (2015). Insights and Recommendations from the MAA National Study of College Calculus. MAA Notes. Washington, DC: Mathematical Association of America (to be available August, 2015).

Donovan, M.S. & Bransford, J.D. (eds.). (2005). How Students Learn: Mathematics in the Classroom. Washington, DC: National Academies Press. www.nap.edu/catalog/11101/how-students-learn-mathematics-in-the-classroom

 Freeman, S. et al. (2014). Active learning increases student performance in science, engineering, and mathematics. Proc. National Academy of Sciences. 111 (23), 8410–8415. www.pnas.org/content/111/23/8410.abstract

Fry, C. (ed.). (2014). Achieving Systemic Change: A sourcebook for advancing and funding undergraduate STEM education. Washington, DC: AAC&U. www.aacu.org/pkal/sourcebook

Hsu, E., Murphy, T.J., Treisman, U. (2008). Supporting high achievement in introductory mathematics courses: What we have learned from 30 years of the Emerging Scholars Program. Pages 205–220 in Carlson and Rasmussen (eds.). Making the Connection: Research and Teaching in Undergraduate Mathematics Education. MAA Notes #73. Washington, DC: Mathematical Association of America. www.maa.org/publications/books/making-the-connection-research-and-teaching-in-undergraduate-mathematics-education

Kober, N. (2015). Reaching Students: What research says about effective instruction in undergraduate science and engineering. Washington, DC: National Academies Press. www.nap.edu/catalog/18687/reaching-students-what-research-says-about-effective-instruction-in-undergraduate

Kogan, M. & Laursen, S.L. (2014). Assessing long-term effects of Inquiry-Based Learning: A case study from college mathematics. Innovative Higher Education 39(3) 183–199. link.springer.com/article/10.1007%2Fs10755-013-9269-9

Calculus at Crisis III: The Client Disciplines

In my last years at Penn State, I worked with faculty in the College of Engineering on issues of undergraduate education. They had two complaints about the mathematics department. First, we were failing too many of their students. Second, the ones we passed seemed incapable of using the mathematics we presumably had taught them when they got to their engineering classes. Chief among their specific gripes was an inability among their students to read a differential equation, to understand its assumptions of the relationships among the quantities being measured.

A decade later, in the Curriculum Foundations Project workshops with engineering faculty brought from across the country, we heard the same concerns about what their students should be learning from the mathematics department:

Students “should understand the reasons for selecting a particular technique develop an understanding of the range of applicability of the technique, acquire familiarity with the mechanics of the solution technique, and understand the limitations of the technique.” (from civil engineers, p. 59)

“There is often a disconnect between the knowledge that students gain in mathematics courses and their ability to apply such knowledge in engineering situations … We would like examples of mathematical techniques explained in terms of the reality they represent.” (from electrical engineers, p. 66)

“In an engineering discipline problem solving essentially mean mathematical modeling; the ability to take a physical problem, express it in mathematical terms, solve the equations, and then interpret the results.” (from mechanical engineers, p. 81)

From the current ABET (Accreditation Board for Engineering and Technology) Criteria for Accreditation, all of the references to mathematics under Curriculum talk about “creative applications,” building “a bridge between mathematics and the basic sciences on the one hand and engineering practice on the other,” and the use of mathematics in the “decision-making process.” As ABET moves into the criteria for specific programs, again the emphasis is entirely on the ability to apply knowledge of mathematics, not on any list of techniques or procedures.

In the biological sciences, the other big driver for calculus enrollments, the American Association of Medical College and the Howard Hughes Medical Institute have dropped the traditional lists of specific courses that students should take in preparation for medical and instead list the competencies that students will need. First among these is mathematics. Of the seven specific objectives within this competency, six speak of quantitative reasoning and the use of data, statistics, modeling, and logical reasoning. The seventh comes closest to calculus, but what they actually ask for is the ability to “quantify and interpret changes in dynamical systems,” a far cry from the usual calculus course. (For more on this report, see my column on The New Pre-Med Requirements.)

In the influential Vision and Change document crafted by the biological sciences with assistance from AAAS, six core competencies for undergraduate biology education are identified. Two of them are mathematical: quantitative reasoning and the ability to use modeling and simulation. The report goes on to specify that “all students should understand how mathematical and computational tools describe living systems.”

These examples can be multiplied in other client disciplines. What we see is a universal need for students to be able to use mathematical knowledge in the context of their own disciplines. In the case of calculus, the challenge is to understand it as a tool for modeling dynamical systems. This is why calculus is required by so many disciplines. But this is an understanding of calculus that is achieved by very few of our students because their focus has been narrowed down to learning how to solve the particular problems that will be on the next exam.

None of this disconnect between what we teach in calculus and the needs of the client disciplines is new. It now rises to the level of a force that is bringing us to crisis because these client disciplines are themselves under the same increased pressure to have their students succeed. There may have been a time when there was a sufficiently rich pool of potential engineers that we could afford the luxury of allowing the mathematics department to filter out all but the most talented, the ones who would succeed in spite of how we taught them. If it ever existed, that time has passed. Our client disciplines now have higher expectations for what and how we teach their students.

Nothing has driven this point home more clearly than Engage to Excel, the Report to the President from his Council of Advisors on Science and Technology (PCAST). (See my columns On Engaging to Excel, Response to PCAST, and JPBM Presentation to PCAST.) The frustration of the scientists in PCAST with calculus instruction that does not meet the needs of their disciplines is evident in their call for “a national experiment [that] should fund … college mathematics teaching and curricula developed and taught by faculty from mathematics-intensive disciplines other than mathematics, including physics, engineering, and computer science.” (Recommendation 3-1, p. vii)

While there was one particular physicist who was the driver behind this report, it did reflect the concerns of all of PCAST’s members. These are scientists and leaders in technology who deplored the fact that “many college students … often are left with the impression that the field [of mathematics] is dull and unimaginative.” (p. 28)

I have yet to find physicists, engineers, or computer scientists who want to take over our calculus instruction. They have better things to do. But some have been forced to do so, and others are contemplating undertaking it as a necessary correction to mathematical instruction that is not meeting their needs.

This completes my triad of forces that constitute the reason we are at crisis. It is the nature of a crisis that the solution is not readily apparent. Nevertheless, there are actions that can be taken to improve the situation. Next month, I will explore the first of these: drawing on knowledge of best practices for effective teaching and learning.

Calculus at Crisis II: The Rush to Calculus

I began this series last month by explaining how recent economic conditions are sending more students into the primary STEM fields (engineering and the physical, biological, mathematical, and computer and information sciences) while constricting the resources available to meet the needs of educating them. This is just one of a triad of phenomena that are pushing college calculus toward crisis. This month, I will discuss the second of these forces: the rush to calculus.

Nothing illustrates the relentless growth of high school calculus better than the graph of the number of AP Calculus exams taken each year (Figure 1), surpassing 400,000 in 2014. According to NCES data [1], 53% of the students who study calculus in high school take an AP Calculus exam, implying that roughly 750,000 U.S. high school students studied calculus this past year. By comparison, this past year only 250,000 students took their first mainstream calculus class at a 2- or 4-year college or university [2].

Figure 1: Total AP Calculus exams and fall enrollments in mainstream Calculus I. Sources: The College Board and CBMS Statistical Abstracts.

What happens to the students who study calculus in high school? We know from the MAA study Characteristics of Successful Programs in College Calculus that about one- third of them retake Calculus I when they get to college. Based on AP scores and common policies for granting credit, roughly 200,000 students accept credit and/or advanced placement for their high school work. From a limited study [3], a clear majority of these students, probably three-quarters or more, do continue on to further courses that build on calculus.

All of these patterns intensify at research universities and elite colleges, where at least 70% of Calculus I students are retaking a course they have already seen in high school, and large numbers of students heading for math-intensive majors skip over Calculus I.

The result has been a dramatic change in the make-up of Calculus I. At most colleges and universities, it makes little sense to teach this course as if students are encountering calculus for the first time; few of them are. It also makes little sense to teach this course as if the students are heading into the mathematical or physical sciences. Nationwide, only 6% of Calculus I students intend such a major [4]. Finally, it makes little sense to teach this course as if this is where we see our best-prepared students.

This last point is clear if we consider how many of the best-prepared students skip Calculus I, but it also is a consequence of what the rush to calculus has done to the middle and high school curricula in mathematics.

In fall 2014 there were just over 1.6 million full-time first-year students enrolled in 4- year undergraduate programs in the U.S. [5]. Assuming that most of the 750,000 who take calculus in high school are traditional college-bound students who will enroll as full- time students in 4-year programs, these high school calculus students will constitute 40–45% of traditional first-year college students. The result is a common belief among parents, guidance counselors, and administrators that every college-bound student should, if at all possible, study calculus before high school graduation. I hear this from college students whose reason for taking calculus in high school was that it was expected of their peer group, and I hear it especially from high school teachers who complain of the tremendous pressure they are under to expand calculus classes and admit students they know are not ready for it.

Because high school calculus by itself has become such common coin, those students who aspire to an elite college or university try to take calculus, preferably BC Calculus, before 12th grade. Figure 2 shows the exceptional growth in the number of students who take an AB or BC Calculus exam before grade 12.

Figure 2: Number of AP Calculus exams taken by students in grade 11 or earlier. Source: The College Board.

We do not know the full effect of this movement of calculus into ever earlier grades, but there is strong anecdotal evidence from teachers at both the high school and university level that many of these students are short-changing their preparation in middle and high school mathematics to join the fast track to calculus. Again anecdotally, this appears to be a significant problem when students attempt a math-intensive major where weaknesses in precalculus material can be disastrous.

We can deplore the rush to calculus in high school, but the forces that are sustaining it are formidable. We have neither the authority nor the certain knowledge that would enable us to halt or reverse it. For the foreseeable future, we will have to live with it.

Just in the past ten years, the preparation and aspirations of our college calculus students have shifted significantly. We cannot afford to assume that curricula and methods of instruction that were sufficient for the past will be adequate for the future.


[1] National Center for Education Statistics (NCES). (2012). An overview of classes taken and credits earned by beginning postsecondary students. NCES 2013-151rev. Washington, DC: US Department of Education. nces.ed.gov/pubs2013/2013151rev.pdf

[2] By “mainstream” we mean a calculus course that can be used as part of the pre- requisite stream for more advanced mathematics courses. It usually does not include business calculus, but may or may not include calculus for biologists. The figure of 250,000 is an estimate based on data from the CBMS Statistical Abstracts and the MAA study Characteristics of Successful Programs in College Calculus. Approximately 500,000 students began mainstream Calculus I at the post-secondary level at some point in the past year, and roughly half of them had studied calculus in high school.

[3] Morgan, K. (2002). The use of AP Examination Grades by Students in College. Paper presented at the 2002 AP National Conference, Chicago, IL.

[4] Source: MAA National Study of College Calculus, www.maa.org/cspcc.

[5] Source: HERI, The American Freshman. www.heri.ucla.edu/tfsPublications.php

Calculus at Crisis I: The Pressures

By David Bressoud

Crisis: A decisive moment. The choice of preposition in the title of this new series is intentional. To be “in crisis” indicates a desperate situation that is not sustainable. I have chosen “at crisis” to indicate a degenerating situation that calls for decisive change. I will begin this series with an account of some of the pressures that have brought us to this pass.

In February of 2012, the President’s Council of Advisors on Science and Technology (PCAST) produced a report, Engage to Excel [1], that called for an additional one million majors in Science, Technology, Engineering and Mathematics (STEM) over the next ten years (see also my column On Engaging to Excel, March 2012). By coincidence, that spring there was a 7.5% increase over the previous year in the number of Bachelor’s degrees awarded in five primary STEM disciplines: Engineering as well as the Biological, Physical, Computer, and Mathematical Sciences. Obviously, this had little to do with PCAST’s wishes.

As I pointed out many years ago [2], economic considerations drive much of how students choose their field of study. Many if not most of those 2012 graduates had arrived in college in the economically momentous fall of 2008. In fact, beginning with the class that entered in fall 2008, there has been a sharp and continuing increase in the number of students who come to college with the intention of pursuing a STEM degree (Figure 1).

Figure 1. Number of full-time first-year students in four-year undergraduate programs

who intend to major in the designated field. Dashed line at 2007.

Source: HERI [3].

From 2007 to 2008, the number of entering students intending to major in Engineering rose by 32.5%. From 2007 until this past fall, the number of freshmen heading into any of these STEM fields rose by 92%, from 276,000 to 531,000.

There is, of course, a four to six year lag between matriculation and graduation. It is still early to assess the full impact of the increased interest in STEM that began in 2008. Figure 2 compares the number of entering freshmen in a given year who intend to major in one of the five primary STEM fields with the number who received a Bachelor’s degree in one of those disciplines in that year.

Figure 2. Number of entering freshmen intending to major in one of the given primary STEM disciplines versus the number of Bachelor’s degrees awarded in these disciplines.

Source: HERI [3] and NCES [4].

It is interesting to observe that, starting in 2009 to 2010, annual growth in the number of STEM degrees switched from 1 to 2% per year up to 5 to 7% per year. This growth started before the class that entered in 2008 could have graduated and may reflect recognition of the value of staying in a STEM major. The large jump in intended majors from 2007 to 2008 is not reflected in a comparably large jump in the number of degrees from 2012 to 2013. Part of this is probably due to the fact that an engineering degree is often a five-year degree. But it also almost surely reflects the fact that a large increase in the number of students seeking such a degree will include a significant number of students who are only marginally qualified to successfully complete this degree.

Implications. Enrollment in introductory STEM courses is driven by incoming students. This is especially true for Precalculus and Calculus I. Unfortunately, the same economic pressures that are pushing more students into STEM fields are forcing staff reductions in our universities. At the same time that more of the marginally prepared students are seeking STEM degrees, more of the best prepared students are using Advanced Placement® and other credits to skip over these introductory courses. And financially strapped states are requiring greater accountability for the dollars given to their universities, mandating higher success rates.

It is a perfect storm: University mathematics departments are required to teach greater numbers of students who are less well-prepared, using fewer resources and with increased expectations for student success. These alone would be sufficient to warrant the designation “at crisis.” In fact, much more is now forcing us toward change, including the rush to calculus in high school and changing demands of the client disciplines as illustrated in PCAST’s Engage to Excel. Over the next several months I will describe these challenges and what it will take to meet them.

References

[1] President’s Council of Advisors on Science and Technology (PCAST). 2012. Engage to Excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-engage-to-excel-final_feb.pdf

[2] David Bressoud. 2010. A Benefit of High Unemployment, MAA Launchings, www.maa.org/external_archive/columns/launchings/launchings_11_10.html

[3] Higher Education Research Institute (HERI). 2007 through 2014. The American Freshman: Forty Year Trends and The American Freshman. Los Angeles, CA: HERI, UCLA. www.heri.ucla.edu/tfsPublications.php

[4] National Center for Education Statistics (NCES). 2002 through 2014. Digest of Education Statistics. Washington, DC: U.S. Department of Education. nces.ed.gov/programs/digest/

Reaching Students

By David Bressoud

The National Academies have just released a report that should be of interest to readers of this column: Reaching Students: What research says about effective instruction in undergraduate science and engineering. [1] It is based on their earlier report, Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering (the DBER Report), which was the subject of my Launchings column in December 2012, Mathematics and the NRC Discipline-Based Education Research Report. The new report illustrates the insights and recommendations from DBER with current examples and presents practical suggestions for improving classroom instruction.

Before I get into the many things I like about this report, I will start with its one glaring fault: It completely ignores undergraduate mathematics education. Like the DBER Report itself, it reads as if mathematicians have never thought about effective classroom practice. Based as it is on the DBER Report, this is perhaps not surprising. It is still disappointing.
Nevertheless, there is a lot that mathematicians can learn from this report. The many examples that describe actual classroom practice include:

• Facilitation of reflective learning (p. 6)
• Use of peer-led team learning (p. 18)
• Effective use of clickers in large classes (p. 22)
• Effective use of learning goals (p. 37)
• Methods for identifying the ideas that are most misunderstood by or confusing to students (p. 67)
• Assessment in active learning classes (p. 124)
• Effective faculty professional development (p. 196)
• The Association of American Universities efforts to improve undergraduate STEM education (p. 203)

These call-out illustrations are interspersed among pointed and helpful discussion of the issues faced by those who are working to improve undergraduate STEM education. It starts with the basics: how to find like-minded colleagues, how to find resources, and the benefits of joining a learning community.

This report discusses the role of lecturing, both its strengths and its weaknesses. More importantly, it talks about strategies for making lectures more interactive. It looks at assessment as more than measuring what questions students can answer, describing how to use it—especially student writing—to understand student reasoning, misconceptions, and misunderstandings.

It also deals with the challenges of changing one’s pedagogy and the obstacles that we all face, recognizing the difficulty in finding the time and energy required to adapt one’s approach to teaching. The advice includes: start with whatever is comfortable for you, use proven materials that others have developed, take advantage of the support that is available (there are many small grants specifically designed to ease the adoption of such practices [2]), and share the effort with interested colleagues.

The report also tackles the issue of coverage, one of the most frequently cited reasons for sticking with lectures. As the report accurately states, “What really matters is how much content students actually learn, not how much content an instructor presents in a lecture.” (p. 160) Moreover, as I have found in my classes, helping students learn how to think about mathematics, how to read it, how to wrestle with it, how to tackle unfamiliar and challenging problems, means helping them learn how to learn it on their own. As we succeed in these goals, there will much content that can be assigned to them to learn through reading or online resources rather than by taking up precious contact time.

Noah Finkelstein of CU-Boulder makes exactly this point, “You must be willing to move away from the idea that teaching is the transmission of information and learning is the acquisition of information, to the notion that teaching and learning are about enculturating people to think, to talk, to act, to do, to participate in certain ways.” (p. 31)

This enculturation enables students to use what they have learned in our classes. As the report states in the chapter on Using Insights from Research on Learning to Inform Teaching, “expertise consists of more than just knowing an impressive array of facts. What truly distinguishes experts from novices is experts’ deep understanding of the concepts, principles, and procedures of inquiry in their field, and the framework for organizing this knowledge.” (their italics, p. 58)

Helping students develop this kind of expertise is difficult, but we know that active learning approaches are much more effective than simply watching an expert produce the solution in a flawless flow.

The report ends with a summary of lessons (pp. 212–213), from which I have chosen and paraphrased four:

1. Begin by understanding how students learn. [3]
2. Start small with the changes that make the most sense and are easily implemented.
3. Establish challenging goals for what students will learn and use them to guide both your instructional strategies and your assessments.
4. Draw on the research, materials, and support structures that are already available.

I hope that this report will sit in the reading room of every math department and at hand for every mathematician who cares about teaching.

References:

[1] Kober, N. (2015). Reaching Students: What research says about effective instruction in undergraduate science and engineering. Washington, DC: The National Academies Press. www.nap.edu/catalog/18687/reaching-students-what-research-says-about-effective-instruction-in-undergraduate
[2] One example of a source of small grants for the teaching of undergraduate mathematics is the Academy of Inquiry Based Learning, www.inquirybasedlearning.org.
[3] Two of the best resources for this are:
Ambrose, S.A., Bridges, M.W., DiPietro, M., Lovett, M.C., and Norman, M.K. (2010). How Learning Works: Seven research-based principles for smart teaching. San Francisco, CA: Jossey-Bass.
National Research Council. (2005). How Students Learn: Mathematics in the Classroom. M.S. Donovan and J.D. Bradford, Editors. Washington, DC: The National Academies Press.

The Emporium

By David Bressoud

Last month, in MOOCs Revisited, I looked at one version of the use of online resources. This month I’d like to comment on another approach to using technology to improve student learning while cutting costs, the Math Emporium, first adopted on a large scale at Virginia Tech. It shifts math classes from large lecture halls to computer labs where students are required to put in a certain number of hours each week in which they work through computer supplied problems while wandering tutors help those in difficulty.

My column is inspired by a visit I made in February to a large public university that uses a Math Emporium for their pre-calculus courses: Intermediate Algebra, College Algebra, Trigonometry, and Pre-Calculus. Their operation is on a large scale. Just over 4,000 of their students took one of these four courses in fall 2014. This was my first opportunity to observe and probe the workings of a Math Emporium. This column, however, is not about what I found at that particular university. Rather, I am using that experience to reflect on what I see as the strengths, weaknesses, and possibilities of the emporium model.

As I observed the workings of the emporium, I noted four distinguishing characteristics:

Self-pacing. The fact that computers mediate almost all of the learning means that students have a great deal of flexibility in the pace at which they proceed through the course. This was particularly appreciated by returning adult students and those for whom their last mathematics class was in the distance past. For them, it was helpful to be able to work at assignments until they were correct and postpone quizzes until a level of mastery had been achieved.

Compulsory laboratory attendance. In the emporium that I observed, students were required to spend at least three hours per week in the laboratory, a tightly structured environment in which they had access to nothing except their computer, which was locked onto that week’s lessons, videos, homework problems, and quizzes. For three hours a week, there was nothing they could do except work on mathematics. Almost all of the students I talked with chafed at this. They would prefer to do this work in a more personal and relaxed environment. Yet, the fact is that many students, especially those at most risk, do not know how to structure their time effectively. The lab forced a structure on them.

One lesson I took away from the particular emporium I visited was the importance of a welcoming environment within the computer lab. The prospect of being forced to spend time in a sterile, unfriendly room can be a strong disincentive to enrolling in a math course run in the emporium model.

Tutoring. An essential feature of a Math Emporium is the presence of tutors circulating among the working students. Students can use the computer to signal a request for a tutor, but often the interaction happens more informally when a student catches a tutor who happens to be walking by. Moreover, tutors are trained in how to identify students who are struggling and how to offer assistance. Not all students are willing to signal for help.

Help in the laboratory comes from three categories of personnel. There are the instructors responsible for setting the syllabus, homework assignments, quizzes, and exams as well as meeting regularly with the tutors to prepare them for potential student difficulties with the upcoming materials. Spending time in the emporium is part of their responsibilities. There are graduate students, usually in their first year, for whom this is their work assignment. And there are undergraduate students, many of whom also experienced the emporium as students. Talking with students, it is clear that the dedication and abilities of the tutors, especially the graduate students, vary widely.

The particular university I visited continues to run one 50-minute lecture per week for each class of 300 to 400 students. It serves as an introduction to the material but offers little to no opportunity for student/faculty interaction. However, I found that most of the students identified strongly with their instructor and preferred to snag him (none of the instructors are women) when in the emporium. As a helpful feature, the screens are color-coded so that instructors can identify the students in their classes from a distance, and student names are prominently displayed on the screen so that instructors can pretend they know them by name.

This raises an interesting point that I touched on last month: For most students, it is important to have some sense of a personal connection with their instructor. One can question how much benefit students derive from their once-a-week 50-minute meeting with the instructor in the company of 350 other students, but the students with whom I talked did feel some connection to their instructor, strengthened when the instructor would stop to talk with them in the emporium. Many of them chose the time they came to the emporium by when they knew their instructor would be present.

Assessments. Students know that what counts is what is on the test. One of the major drawbacks of purely computer-mediated testing is that the problem format has usually been restricted to multiple choice and short answer questions, a format that enforces a view of mathematics as a collection of procedures to be mastered, with little opportunity for assessing the development of a structured understanding of the undergirding principles.

For the courses at the Math Emporium that I observed, high school courses that many if not most of the students are repeating, there may be a case for instruction focused on one-step procedural fluency. Nevertheless, one of the dominant complaints among the faculty in this Department of Mathematics was that the students enter calculus with little experience in multi-step problem solving or justification of what they have done. Technology is changing what can be assessed, but changing large-scale assessment to capture multi-step problem solving and conceptual understandings is still difficult.

The Math Emporium was created as a response to the reality of teaching large numbers of students with few instructors, combined with the recognition that large lecture classes were not working. Large lecture classes can work, as attested in Frank Morgan’s Huffington Post blog, “Are smaller college calculus classes really better?”. In fact he quotes my observation from the MAA National Study of College Calculus that revealed no correlation between class size and changes in student attitudes. But I think that lack of correlation has more to do with the fact that classes of any size can be taught poorly than that class size is really immaterial. Furthermore, I am unconvinced by Frank’s examples of large lecture classes that work. All of his examples are at institutions with very highly motivated students who know how to study on their own. I also believe that, while 100–120 students constitute a large class, there is a qualitative difference between large classes of this size and classes of 300–400 students where instructors cannot possibly monitor or encourage the performance of more than a small number of their students.

The Math Emporium is far from the ideal of what we would like undergraduate education to be. Unfortunately, that ideal is incredibly expensive. The emporium model does provide a relatively inexpensive means of structuring how students study, monitoring their progress, and providing some degree of individual attention. There is every reason to believe that it provides a framework that can work for many students. Moreover, there are and will continue to be opportunities to improve its effectiveness.

MOOCs Revisited

Despite this month’s title, I have refrained from writing about MOOCs, Massive Open Online Courses, in this column before now. The initial burst of interest always seemed overdone to me. Now that the enthusiasm has waned, we are beginning to see the emergence of meaningful information about when and how they can be useful.

As I argued in my co-authored piece in the AMS Notices, Musings on MOOCs [1], they do seem to hold promise as a source of supplementary material that enables flipped classes, supplementary instruction, alternate approaches, or opportunities for exploring topics that extend beyond the course syllabus. Two questions immediately emerge: How hard is it to take advantage of these materials? Do students actually benefit?

This past summer, Rebecca Griffiths and her team at Ithaka S+R, an academic consulting and research service, released Interactive Online Learning on Campus [2], its study of the use of hybrid MOOCs within the University System of Maryland. Hybrid MOOCs are face-to-face classes for which instructors draw on online courses, in this case developed by Coursera or the Open Learning Initiative, to supplement their own instruction. Griffiths et al. conducted seven side-by-side studies, direct comparisons of the same courses taught with and without these online supplemental materials, and ten case study investigations of courses that were only taught with supplemental materials derived from MOOCs. The side-by-side comparisons are of greatest interest to me because of the usefulness of direct comparisons and because these courses included STEM subjects: three sections of introductory biology and one each of pre-calculus, statistics, and computer science, plus a course in communications.

In answer to the first question—How hard is it to incorporate material from these online courses?—the answer is hard, but probably will become easier when repeated. Griffiths et al. found that self-reported instructor time spent selecting the materials and preparing how they would incorporate them into their hybrid course had a median of 68 and a mean of 144 hours, roughly two to four weeks. The variation was tremendous, from only one full-time week to an entire summer. Most of this is, almost certainly, a one-time investment. For some hybrid courses, face time was reduced by as much as 50%. For others, there was no reduction in face time. Once the start-up time is invested, there appears to be potential for some time—and therefore cost—savings, although it would be modest at best.

The biggest question is whether this improved student outcomes. For the most part in the side-by-side comparisons, there was little difference in pass rates or student performance on a common post test. One biology section had a substantially and significantly better pass rate for the hybrid course, but the other two hybrid biology sections had slightly lower (though not statistically significantly lower) pass rates than the sections with which they were paired. With two exceptions, results on the post tests were indistinguishable between hybrid and non-hybrid courses. Those exceptions were the biology section with the high pass rate and the pre-calculus class. In both of these cases, the hybrid classes posted substantially higher post test results that were significant at p < 0.001.

Griffiths et al. also looked at pass rates and post test results by key subgroups involving race, gender, socio-economic status, and SAT scores. Averaging across all of the side-by-side comparisons for each of the subgroups, pass rates and post test results improved with the hybrid courses, although none of the pass rate differences were significant at p < 0.01. However, several of the post test comparisons were. Although all students saw gains from the hybrid approach, the greatest gains were to White and Asian students, females, and those with parental income between $50,000 and $100,000, at least one parent with a BA, and combined SAT scores above 1000.

There were other factors that came into play. Students preferred the traditional course format and felt that they learned more from it, although they did prefer to do their homework assignments, quizzes, and exams online. Technical glitches did arise in the hybrid courses and may have been a factor in student dislike of online instruction.

One of the most intriguing differences was in how much time students spent on the course outside of classtime. Here the effects were in opposite directions for: under-represented minorities (URM) versus non-URM, low income versus high income, first generation college student versus not first generation, SAT scores below 1000 versus SAT score above 1000. In all cases, the first group saw a decrease in time spent outside of class with the hybrid course, the second group an increase. It may be that online materials allowed students in these traditionally under-represented subgroups to make more efficient use of their time, thus needing to spend less time. But that is a hypothesis that would require study. On its face, this distinction is troubling.

[1] Bonfert-Taylor, P., Bressoud, D.M., and Diamond, H. 2014. Musings on MOOCs. Notices of the AMS. Vol 61, pp. 69–71. www.ams.org/notices/201401/rnoti-p69.pdf

[2] Griffiths, R., Chingos, M., Mulhern, C., and Spies, R. 2014. Interactive Online Learning on Campus: Testing MOOCs and Other Platforms in Hybrid Formats in the University System of Maryland. New York, NY: Ithaka S+R. www.sr.ithaka.org/research-publications/interactive-online-learning-on-campus

The Benefits of Confusion

This past September, The Chronicle of Higher Education published an article with which I strongly resonated, “Want to help students learn? Try confusing them.” [1]. It described an experiment in which two groups of students were each presented with a video of a physics lecture. The first lecture was straightforward, using simple animations and clear explanations. The second involved a tutor and a student in which the student struggled to understand the concepts and the tutor provided leading questions but no answers. Coming out of the videos, students found the first to be clear and easy to understand, the second very confusing. Yet when later tested on this physics lesson, students who had seen the second video demonstrated far more learning than those who had seen the first.

This illustrates the problem with so much of standard instruction, especially in undergraduate mathematics. The ideas have become so polished over decades if not centuries, and we who teach this material understand its nuances so thoroughly, that what we present glides easily past our students without opportunity to grasp its true complexities. For learning to take place, students must engage and wrestle with the concepts we want them to understand.

Cartoon courtesy of brownsharpie.courtneygibbons.org
licensed under the Creative Commons BY-NC-SA 3.0 license

I am not advocating confusion for confusion’s sake. As Courtney Gibbon’s cartoon illustrates, a polished lecture can also be very confusing, and not in a good way. Confusion is most productive when it provides a focus for personal investigation. An example of positive confusion is the cognitive dissonance produced when student expectations confront convincing evidence that they are wrong. My prime example of this is George Pólya’s Let Us Teach Guessing (see my Launchings column Pólya's Art of Guessing).

I like to think of this as “gritty” mathematics rather than confusing mathematics. One of my favorite examples from personal experience was a Topics in Real Analysis course that I taught in Spring 1997 using Thomas Hawkins’ doctoral dissertation, Lebesgue’s Theory of Integration: Its Origins and Development, as the text. My experience teaching that course laid the foundation for my textbook A Radical Approach to Lebesgue’s Theory of Integration. Back in 1998, I wrote a paper about this experience, “True Grit in Real Analysis.” I never published it, but I still like it, and as a New Year’s gift to readers, I offer a link to that paper.


[1] Kolowich, S. Confuse Students to Help Them Learn. The Chronicle of Higher Education. September 5, 2014. Available at http://chronicle.com/article/Confuse-Students-to-Help-Them/148385/