IQ Tests for AI?
I read this article online spectrum.ieee.org/how-do-you-test-the-iq-of-ai and found it interesting enough to share here. It talks about how we can test the humanlike aspects of AI's intelligence such as concept learning and analogical reasoning. The article describes some tests that are being used:
- Generating images from patterns (advanced version of Raven's Progressive Matrices)
- AI has to generate the missing image from scratch
- link to study
Generating Correct Answers for Progressive Matrices Intelligence Tests
- Bongard-LOGO - software-generated version of Bongard Problems
- AI has to determine whether new sketches match the pattern
- link to study
BONGARD-LOGO: A New Benchmark forHuman-Level Concept Learning and Reasoning
- Abstraction and Reasoning Corpus (ARC) - set of visual puzzles that test core human knowledge of geometry, numbers, and physics (link to study)
- AI has to interpret the rules followed by the given grids and then apply the analyzed pattern to complete another grids.
- Kaggle even held a competition challenging participants to develop AI systems that could solve the reasoning tasks from the ARC dataset.
Test-makers hoped to improve current AI tech with these tests.
Evidently, AI struggled at understanding abstract ideas, learning from a few examples, and figuring out how things could fit together. AI requires huge amounts of training data for every new skill we want it to learn making it difficult to demonstrate a core aspect of intelligence which is the ability to learn new skills quickly.
What do you think of these tests?