Comparison of Pre- and Post-semester FCI Scores to Evaluate the Effects of an Online Educational Software on Student Learning

Using the Force Concept Inventory, students within the treatment (n=21) and control (n=23) groups of an engineering mechanics course at UW – Stout were evaluated both prior to treatment and after treatment. The results of this evaluation were interpreted based on the gain for each student as a percentage of maximum possible gain. This metric allows for a more independent analysis among “weaker” and “stronger” students.

The results of this interpretation showed that for the treatment group the average gain was 14.0 % (22.1 % if considering only students with positive net change). Similarly, for the control group the average gain was 5.8 % (15.6 % if considering only students with positive net change). Treatment.png shows the plot of post- versus pre-test scores for the treatment group while control.png presents the same for the control group. Each data point represents a student’s pair of scores while the line corresponds with no change in score. The further a data point is from the line, the larger the gain in that student’s score. It is apparent that for the treatment group, larger gains were realized for students spanning from the lowest scoring students to the highest. For the control group, the majority of the large gains were observed with the “weaker” students while the higher performing students actually scored the same or worse on the post-test. These results would seem to suggest that the use of the system benefited a wider selection of the students in the group while the more traditional approach showed greater gains for the “weaker” students with a possible negative impact on the “stronger” students.