Aristo Developing Skills Answer
Lorri Dent
LinkedIn and 3rd parties use essential and non-essential cookies to provide, secure, analyze and improve our Services, and to show you relevant ads (including professional and job ads) on and off LinkedIn. Learn more in our Cookie Policy.
A lot of us are familiar with tests and exams we have been taken through our life from school exams to college admissions tests to a driving or a corporate ethics test. I was recently taking one of these tests and wondered if a bot or an AI system could take the test and if such an intelligent system exists today. I am particularly referring to tests which require reading, learning, and reasoning. In this article, I will share the latest advancements in AI and analyze the latest QA (Question Answering) systems through an example course and exam.
A typical corporate learning test is often a mix of a few types of questions as shown below and requires the candidate to pass every question in order to pass the exam. Questions are almost always multiple choice varying between a binary Yes/No and many answer choices. Employees can take the test more than once during the same or multiple sittings, but are required to pass it at least once. Broadly, I would categorize the questions in a test like this as one of the following.
Solution: A system to answer these questions correctly might simply need to do smart text or semantic matching between the question and the course content. Using the latest language models and technologies that Google and Bing or AI2 have developed, this should not be a hard problem to solve. Google and Bing already pick up passages from the pages they indexe and show relevant answers for user questions (see screenshot below and read more here). There are many ways to develop an AI system to answer these questions - using latest text matching techniques, training a model on already taken tests, developing systems that can learn and reason.
Choose from the following: (a). Tell Tiara you can have lunch with her but that you would want both her and you to keep the masks on when not putting food in your mouth; (b). Tell Tiara you cannot have lunch with her and stop any communication with her thereafter; . Report Tiara to the Building Management committee
As we can see above, there is a lot of advancements happening across language understanding and text matching, learning and reasoning, and language generation, with the advent of complex models that can train on tons of data and have billions of parameters while running on massive computation power. The use cases and benefits of these new systems are also many from answering simple and complex questions people have every day to writing summaries to curating creative content and many more. From shopping to education to healthcare to entertainment, AI and ML systems will continue to make our lives easier. While this happens, we have to make sure we are aware of any privacy, security, fake information or ethics issues that arise as machines learn more and more tasks are automated.
Aristo, though, is an artificial intelligence program and scientists would like the world to know this is a big deal, as "a benchmark in AI development," as Melissa Locker called it in Fast Company.
We mean, just think about it. Cade Metz, in The New York Times, has thought about it. "Four years ago, more than 700 computer scientists competed in a contest to build artificial intelligence that could pass an eighth-grade science test. There was $80,000 in prize money on the line. They all flunked. Even the most sophisticated system couldn't do better than 60% on the test. AI couldn't match the language and logic skills that students are expected to have when they enter high school."
For the direct story, you should read "From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project," which is now up on arXiv. This project was a six-year mission to answer grade-school and high-school science exams.
The authors were well aware that AI had not made an impressive show in the past of performing on desired levels. With all of AI's mastery at Go, Poker and jeopardy, they said, "the rich variety of standardized exams has remained a landmark challenge. Even in 2016, the best AI system achieved merely 59.3% on an 8th Grade science exam challenge."
Here is the way the AI2 describe its non-human whiz: "Aristo brings together machine reading and NLP, textual entailment and inference, reasoning with uncertainty, statistical techniques over large corpora, and diagram understanding to develop the first "knowledgeable machine" about science."
The team pampered Aristo for an ulterior motive, less to do with patting themselves on the back and more about what they could learn from Aristo's behaviors on science exams, "as these questions test many of the key skills required for machine intelligence," they said.
Stephen Johnson in Big Think wrote about Aristo's inability to do diagrams. He said "the system is designed only to interpret language, meaning it can answer multiple choice questions, but not those featuring an illustration or graph."
For the institute, Aristo's feat is not taken as a perch on the mountain but rather a step in a desired direction. They call it a milestone "on the long road toward a machine that has a deep understanding of science and achieves Paul Allen's original dream of a Digital Aristotle."
c80f0f1006