Evaluation of the effectiveness of CAI modules in mathematics for intermediate grade schools

Added Title

Evaluation of courseware in mathematics

College

Br. Andrew Gonzalez FSC College of Education

Department/Unit

Science Education

Document Type

Report

Publication Date

7-31-2014

Abstract

This study evaluated the effectiveness of the Computer-Aided Instruction (CAI) courseware modules in Mathematics developed by the Department of Science and Technology – Science Education Institute in collaboration with the Advanced Science and Technology Institute (ASTI) in terms of motivating pupils in Mathematics learning and improving their performance in an achievement test. It sought to determine specifically (1) the extent of CAI modules use in Math classes; (2) teachers’ perception of the CAI modules; (3) pupils’ perception of the CAI modules; (4) if there is significant improvement in pupils’ Mathematics performance brought about by the CAI modules; (5) if there is significant relationship between the CAI’s motivational design and its Mathematics content as reflected in pupils’ performance; and (6) the extent of pupils’ engagement in using the CAI modules.

Twenty-five (25) schools in 3 divisions of the Metropolitan area were surveyed for their demographic profile. This consisted of 86 teachers from grades 4, 5 and 6. Eighteen (18) willing teachers and 843 pupils went through the CAI modules, after which they were administered a survey questionnaire based on Keller’s (2006) ARCS model of Instructional Materials Motivation. A pre-test was administered to pupils prior to and a post-test after they went through the CAI modules. The ratio of pupil to computer unit as pupils go through the CAI learning module during the conduct of this study was 1:1. Responses were analyzed using descriptive statistics. One-way Analysis of Variance (ANOVA) was used to determine if there is significant difference among the three grade level pupils’ perception in each interactivity category. A paired samples t-test was used to see if there is a significant difference between pupils’ pre-CAI and post-CAI achievement test scores in the selected CAI topics. Correlation between pupils’ responses on motivational design and their performance in the achievement test was used to see which modules have significant relationship in the motivational aspects. Pupils’ attitude towards using the CAI modules was culled from their response when they were asked to comment about the CAI modules. Thirty randomly selected pupils were observed while they were going over the CAI modules. Facial expressions, mouse strokes and keyboard responses were recorded through installed web cameras and an application called Observatory.

Findings include but not limited to the following:

1. Only a limited number of schools have the capacity to facilitate the use of the Math and Science courseware modules in their instruction due to their physical facilities such as computer laboratories and internet access, as well as teachers’ computer literacy skill.

2. Surveyed teachers’ reason for not using the CAI was due to the lack of orientation and training of the said modules.

3. Teachers agree that the CAI has management interactivity. The pupils have greater control over their learning which makes them involved and motivated to proceed with the module.

4. Teachers strongly agree that the CAI has motivation interactivity. The CAI has features that affect the degree of motivation in the instructional interactivity. E.g. pupils are provided by the CAI with knowledge of results (scores) in drill and practice. In most modules, appropriate feedback is evident in the exercises. The CAI’s background sound is soft and appealing. Role-plays, games, and simulation scenarios are realistic.

5. Pupils perceived that the CAI has the ability to draw their attention as it provides extrinsic and intrinsic reinforcement for effort; topics are relevant, can arouse and sustain curiosity as well as interest, and can build their confidence; and overall module give satisfaction in learning mathematics.

6. Pupils in the higher grade levels better perceived the modules in all aspects of motivational design.

7. There was no enough basis to conclude that there was an improvement in mathematics performance across all topics among Grade 4 pupils who went through the CAI. But there was significant improvement across all topics among Grade 5 pupils except in the topic “Area of Trapezoid”. The effect sizes are small (Cohen, 1988), indicating around 20% to 30% that can be attributed to CAI intervention. There was also significant improvement across all topics in Grade 6. The effect sizes vary across topics. In Measurement, and Surface Area of Square Pyramid, the effects are of medium size ranging from 40% to 65%. The topics Space Figures, Surface Areas, and Volume of Cube and Rectangular Prism, have small effect sizes. Overall, there is a medium effect size attributed to the CAI intervention.

8. There is variability in the impact of motivational design to pupils’ performance. Although there was no enough evidence to establish the relationship of the motivational aspects with the scores of grade 4 pupils, the Attention and Satisfaction aspects prove to be a differing factor (directly but weak) for grade 5 pupils’ scores in two identified modules, and the Confidence aspect prove to be a differing factor (directly but weak) for grade 6 pupils’ scores also in two identified modules.

9. Majority of the students expressed positive feelings, attitude and descriptions of their experience with the CAI modules.

10. No strong statistical significance was found to relate performance or learning gain with pupils’ engagement (interest or attentiveness to the CAI modules). More pupil participants may be needed to give more reliable results to correlate engagement with learning gain.

11. The CAI is highly successful in capturing the interest of the pupils into participating actively in CAI especially during the exercises. Possible reasons are the interactive elements found in exercises and the challenge presented to the pupils. However, the effectiveness of the CAI is hampered by distractive behaviour in the environment, such as conversations with peers. The lesson phase, while critical in imparting important knowledge of the topic, has been found to elicit the least interest. Positive engagement, for the lesson components of the software, is more likely affected by the conditions of the learning environment than the CAI itself.

Recommendations:

Teachers need to be oriented on the CAI and its proper use. Schools’ infrastructure and facilities have to be made available to enable the use of CAI. If a school does not have an internet access, copies of the modules in CDs must be provided.

Since the current CAI modules based on the pupils’ evaluation are already Satisfying, focus should be directed towards the other aspects of Attention, Relevance and Confidence as these four aspects of the ARCS model should complement each other in motivating a pupil.

Specific aspects of specific modules that need to be improved or refined can be patterned after the exemplary modules that exhibited best features. Specific focus should be given to redeveloping the grade 4 modules that proved to have no significant effect in pupils’ Mathematics performance .

In view of these, the following are suggested:

1. Consistent placement of help and main menu buttons for ease of use;

2. Consistent in the way feedback to pupils’ answers are given;

3. Inclusion of an option to adjust background sound;

4. Sequence examples and exercises from easy to difficult;

5. As an assessment principle, a competency must not be tested if not taught in the module;

6. Consistent in giving feedback;

7. Maximize the use of technical features for learning;

8. Ensure correct timing of voice over;

9. Avoid putting elements that may distract pupils from the essentials of learning; E.g. a male character has long hair;

10. Legibly display texts and illustrations;

11. Reduce the information presented in a frame;

12. Emphasize important words;

13. Allow pupils to move through information one at a time for focus;

14. Consider using real-life pictures be included, not just cartoon style;

15. Consider adding a glossary for hard to understand words;

16. Ensure all answers in the exercises given be correct;

17. Offer optional challenge questions for the exercises; and

18. Ensure an environment conducive for pupils to be engaged in CAI by minimizing possible distractions.

It is envisioned that through this project, there will be a systematic way of evaluating future similar projects on developing CAI learning modules. ASTI people or anyone who will be contracted by DOST-SEI to develop similar projects become knowledgeable on the standards they have to meet. The CAI learning modules pupils and teachers will use in the future that has undergone the system of evaluation, are better in terms of quality and its impact in the positive gains of students’ learning and achievement in Mathematics.

html

Disciplines

Educational Technology | Science and Mathematics Education

Note

Running title: Evaluation of courseware in mathematics

Keywords

Mathematics—Computer-assisted instruction; Computer-assisted instruction

Upload File

wf_no

This document is currently not available here.

Share

COinS