Kavli Affiliate: Avi Shporer
| First 5 Authors: Iddo Drori, Sunny Tran, Roman Wang, Newman Cheng, Kevin Liu
| Summary:
We demonstrate that a neural network pre-trained on text and fine-tuned on
code solves Mathematics problems by program synthesis. We turn questions into
programming tasks, automatically generate programs, and then execute them,
perfectly solving university-level problems from MIT’s large Mathematics
courses (Single Variable Calculus 18.01, Multivariable Calculus 18.02,
Differential Equations 18.03, Introduction to Probability and Statistics 18.05,
Linear Algebra 18.06, and Mathematics for Computer Science 6.042), Columbia
University’s COMS3251 Computational Linear Algebra course, as well as questions
from a MATH dataset (on Prealgebra, Algebra, Counting and Probability, Number
Theory, and Precalculus), the latest benchmark of advanced mathematics problems
specifically designed to assess mathematical reasoning. We explore prompt
generation methods that enable Transformers to generate question solving
programs for these subjects, including solutions with plots. We generate
correct answers for a random sample of questions in each topic. We quantify the
gap between the original and transformed questions and perform a survey to
evaluate the quality and difficulty of generated questions. This is the first
work to automatically solve, grade, and generate university-level Mathematics
course questions at scale. This represents a milestone for higher education.
| Search Query: ArXiv Query: search_query=au:”Avi Shporer”&id_list=&start=0&max_results=10