When is Assistance Helpful to Learning? Results in Combining Worked Examples and Intelligent Tutoring

,


Introduction
The Assistance Dilemma [1] characterizes a long-standing unsolved problem in the learning sciences: when should instruction provide students with assistance and when should it be withheld?Some researchers have argued for providing maximal assistance [e.g., 2], while others, argue for minimal assistance [e.g., 3].In three studies in the domain of chemistry we have explored the assistance dilemma [4,5].
In this paper we discuss our results in experimenting with an intelligent tutor supplemented with worked examples, a combination that has only recently been investigated.The worked example principle, as stated in [6], is: "Replace some practice problems with worked examples", i.e., provide students with an alternating combination of worked examples and problems.The theory behind the principle is that human working memory, which has a limited capacity, is taxed by strictly solving problems, which requires thinking, such as the setting of subgoals.The rationale, then, is that worked examples free mental resources for learning processes.
But then why mix worked examples and problem solving, as suggested by the worked example principle?The theory seems to suggest that worked examples alone, a high-assistance approach, would be best for learning.Past empirical results have been mixed on this issue.For instance, Trafton and Reiser compared problem solving with no tutoring to interleaved worked examples and problem solving with no tutoring and found statistically significant learning gains and learning efficiency for the alternating condition [7].Lovett performed a study that showed that both a low and high assistant approach could be beneficial [8], while Kalyuga et al's results suggest that assistance should decline over time, as subjects gain expertise [9].These (and other) mixed results indicate that there is room for continued studies to explore the issue of when assistance is appropriate in instruction.
Furthermore, until recently there had been little study of the comparative learning benefits of intelligent tutored problem solving and other forms of assistance.Tutored problem solving is a mid-level assistance approach that provides more assistance than untutored problem solving but somewhat less than worked examples.Only Schwonke et al [10], besides ourselves, have explored the combination of tutored problems and worked examples (without one approach being used and subservient to the other).Our hypothesis is that the interleaving of worked examples with tutored problems will further improve learning beyond the benefits of the tutor itself.We explored this hypothesis using the Stoichiometry Tutor, a web-based intelligent tutor that provides support for a basic subarea of high school chemistry [4,5].

Study Design and Results
In all three studies a 2x2 between-subjects design was employed.The independent variable of primary interest to this paper is Worked Examples, with one level being Tutored Alone and the other Worked Examples + Tutored.In the former condition, which will be referred to as the "Problems Condition", subjects solely solved problems with the Stoichiometry Tutor.In the latter condition, which will be referred to as the "Examples Condition", subjects alternated between studying a worked example (followed by self-explanation questions) and solving an isomorphic problem with the aid of the tutor.(The second independent variable of the studies, "personalization," will not be discussed in this brief paper, as it is not our focus here.) All instructional materials were provided via the Internet.All subjects worked on 10 study problems (15 in Study 1), presented according to the two conditions described above.All subjects were also given pre-and post-questionnaires, and pre and posttests, with all pre and posttest problems involving the same type of problems as the study problems.Instructional videos on stoichiometry were interspersed, as appropriate, with the study problems in both conditions.The N of the 3 studies was, respectively, 63, 60, and 81, for a total of 204 participants across all three studies.

* -Significant result
The results were as follows.The students exhibited significant learning between the pre and posttests in all conditions of all studies.On the other hand, the students in the Examples Conditions did not learn more than those in the Problems Conditions, contrary to previous findings such as [7].However, subjects in all of the Examples Conditions spent significantly less time with the study problems.That is, the subjects in the Examples Condition, while they did not learn more, they learned more efficiently than those in the Problems Condition, as shown in Table 1.

Conclusion
The students in the Examples Condition used 21% less time to complete the same problems.If these results were to scale across a 20-week course, students could save 4 weeks of time -yet learn just as much.What explains these findings?Students in the Examples Conditions worked significantly faster on the first of the isomorphic example-problem pairs (i.e., the example) than the second (i.e., the problem).The extra time the students in the Problems Condition took on the first problems -which often seemed be used to turn problems into examples by clicking to bottom-out hints -did not benefit them.This may be because clicking through hints is a less efficient way to see an example compared to seeing that example immediately.
Our studies, unlike most prior studies, involved tutored instead of untutored problem solving.Like Schwonke et al, our results are consistent with the hypothesis that mid-level assistance provides the greatest learning advantages.We plan to test this more explicitly in a new study, which will compare three levels of assistance: all tutored problems (lower assistance), alternating examples and tutored problems (midlevel assistance), and all worked examples (higher assistance).

Table 1 .
Average learning efficiency, calculated, per subject, as z-score (learning gain) -zscore (instructional time) with z-score = (value -average) / stddev.The P-value was calculated using a one-way ANOVA between the Examples and Problems Conditions' learning efficiency.