IJRUME: Peer-Assisted Reflection

You can now follow me on Twitter @dbressoud.

The second paper I want to discuss from the International Journal of Research in Undergraduate Mathematics Education is a description of part of the doctoral work done by Daniel Reinholz, who earned his PhD at Berkeley in 2014 under the direction of Alan Schoenfeld. It consists of an investigation of the use of Peer-Assisted Reflection (PAR) in calculus [1].

Daniel Reinholz. Photo Credit: David Bressoud

PAR addresses an aspect of learning to do mathematics that Schoenfeld refers to as “self-reflection or monitoring and control” in his chapter on “Learning to Think Mathematically” [4]. As he observed in his problem-solving course at Berkeley, most students have been conditioned to assume that when presented with a mathematical problem, they should be able to identify immediately which tool to use. Among the possible activities that students might engage in while solving a problem—read, analyze, explore, plan, implement, and verify—most students quickly chose one approach to explore and then “pursue that direction come hell or high water” (Figure 1).

Figure 1: Time-line graph of a typical student attempt to solve a non-standard problem.
Source: [4, p.356, Figure 15-3]

In contrast, when he observed a mathematician working on an unfamiliar problem, he observed all of these strategies coming into play, a constant appraisal of whether the approach being used was likely to succeed and a readiness to try different ways of approaching the problem. He also found that mathematicians would verbalize the difficulties they were encountering, something seldom encountered among students (Figure 2). Note that over half the time was spent making sense of the problem rather than committing to a particular direction. Triangles represent moments when explicit comments were made such as “Hmm, I don’t know exactly where to start here.”

Figure 2: Time-line graph of a mathematician working on a difficult problem.
Source: [4, p.356, Figure 15-4]

In the conclusion to this section of his chapter, Schoenfeld wrote, “Developing self-regulatory skills in complex subject-matter domains is difficult.” In reference to two of the studies that had attempted to foster these skills, he concluded that, “Making the move from such ‘existence proofs’ (problematic as they are) to standard classrooms will require a substantial amount of conceptualizing and pedagogical engineering.”

One of the problems with the early attempts at instilling self-reflection was the tremendous amount of work required of the instructor. Reinholz implemented PAR in Calculus I, greatly simplifying the role of the instructor by using students as partners in analyzing each other’s work. The study was conducted in two phases over two separate semesters in studies that each semester included one experimental section and eight to ten control sections, all of whom used the same examinations that were blind-graded. There were no significant differences between sections in either student ability on entering the class or in student demographics. The measure of success was an increase in the percentage of students earning a grade of C or higher. In the first phase of the study, the experimental section had a success rate of 82%, as opposed to the control sections where success was 69%. In the second phase, success rose from 56% in the control sections to 79% in the experimental section.

Reinholz observed a noticeable improvement in student solutions to the PAR problems after they had received peer feedback. From student interviews, he found that many students in the PAR section had learned the importance of iteration, that homework is not just something to be turned in and then forgotten, but that getting it wrong the first time was okay as long as they were learning from their mistakes. Students were learning the importance of explaining how they arrived at their solutions. And they appreciated the chance to see the different approaches that other students in the class might take.

What is most impressive about this intervention is how relatively easy it is to implement. Each week, the students would be given one “PAR problem” as part of their homework assignment. They were required to work on the problem outside of class, reflect on their work, exchange their solution with another student and provide feedback on the other student’s work in class, and then finalize the solution for submission. The time in class in which students read each other’s work and exchanged feedback took only ten minutes per week: five minutes for reading the other’s work (to ensure they really were focusing on reasoning, not just the solution) and five minutes for discussion.

The difficulty, of course, lies in ensuring that the feedback provided by peers is useful. Reinholz identifies what he learned from several iterations of PAR instruction. In particular, he found that it is essential for the students to be explicitly taught how to provide useful feedback. By the time he got to Phase II, Reinholz was giving the students three sample solutions to that week’s PAR problem, allowing two to three minutes to read and reflect on the reasoning in each, and then engaging in a whole class discussion for about five minutes before pairing up to analyze and reflect on each other’s work.

Further details can be found in [2] and [3]. For anyone interested in using Peer-Assisted Reflection, this is a useful body of work with a wealth of details on how it can be implemented and strong evidence for its effectiveness.

References

[1] Reinholz, D.L. (2015). Peer-Assisted Reflection: A design-based intervention for improving success in calculus. International Journal of Research in Undergraduate Mathematics Education. 1:234–267.

[2] Reinholz, D. (2015). Peer conferences in calculus: the impact of systematic training. Assessment & Evaluation in Higher Education, DOI: 10.1080/02602938.2015.1077197

[3] Reinholz, D.L. (2016). Improving calculus explanations through peer review. The Journal of Mathematical Behavior. 44: 34–49.

[4] Schoenfeld, A.H. (1992). Learning to think mathematically: problem-solving, metacognition, and sense-making in mathematics. Pp. 334–370 in Handbook for Research in Mathematics Teaching and Learning. D. Grouws (Ed.). New York: Macmillan.