Document Types
Paper Presentation
School Code
N/A
School Name
Unida Christian Colleges
Research Advisor (Last Name, First Name, Middle Initial)
Cristobal, Joash Raend C.
Abstract/Executive Summary
Over the years, several studies have explored AI's ability in terms of education. This study aims to investigate how ReCo.ai performs in terms of accuracy in answering mathematical concepts about Grade 10 Mathematics. Using the Generative Pre-trained Transformer 3 (GPT-3) model to create a chatbot specifically within the grade 10 Mathematics curriculum, and used the GPT-3 Generated Text dataset for machine learning and natural language processing. The researchers tested the accuracy by evaluating its accuracy, precision and recall, and F1 score. F1 score test was used as a performance metric to evaluate the precision and recall of the chatbot's responses. Furthermore, the researchers analyzed the collected data using Descriptive Statistics (mean and standard deviation). With an accuracy of 61.6% (50-item test) and 78% (10-item test) showed a promising potential for use in educational settings. However, the consistency of accuracy may vary between the topics. This study shows that the chatbot has significant potential for answering grade 10 Mathematics questions, but further research is required to improve its performance and apply it in an actual learning environment. The results of this study have the potential to contribute to the development of intelligent tutoring systems and personalized education technology that can enhance the students' learning experiences in Mathematics and beyond.
Keywords
artificial intelligence; machine learning, chatbot, mathematics, natural language processing
Research Theme (for Paper Presentation and Poster Presentation submissions only)
Computer and Software Technology, and Robotics (CSR)
Initial Consent for Publication
yes
Start Date
26-6-2023 1:00 PM
End Date
27-6-2023 3:00 PM
ReCo.ai: Using Generative Pre-trained Transformer 3 Model for a Chatbot in Answering Grade 10 Mathematics Questions
Over the years, several studies have explored AI's ability in terms of education. This study aims to investigate how ReCo.ai performs in terms of accuracy in answering mathematical concepts about Grade 10 Mathematics. Using the Generative Pre-trained Transformer 3 (GPT-3) model to create a chatbot specifically within the grade 10 Mathematics curriculum, and used the GPT-3 Generated Text dataset for machine learning and natural language processing. The researchers tested the accuracy by evaluating its accuracy, precision and recall, and F1 score. F1 score test was used as a performance metric to evaluate the precision and recall of the chatbot's responses. Furthermore, the researchers analyzed the collected data using Descriptive Statistics (mean and standard deviation). With an accuracy of 61.6% (50-item test) and 78% (10-item test) showed a promising potential for use in educational settings. However, the consistency of accuracy may vary between the topics. This study shows that the chatbot has significant potential for answering grade 10 Mathematics questions, but further research is required to improve its performance and apply it in an actual learning environment. The results of this study have the potential to contribute to the development of intelligent tutoring systems and personalized education technology that can enhance the students' learning experiences in Mathematics and beyond.
