How to Measure Intelligence Beyond Human Scale?
Elad Hazan
Abstract
Based on the academic paper: Measuring Intelligence Beyond Human Scale [1] TLDR: Historically, intelligence benchmarks have been composed of human-generated questions. However, current techniques do not scale to AI, as capabilities surpass human intelligence. We propose the paradigm of adversarial psychometrics , in which participants generate questions and are rewarded for separating each other’s capabilities, without requiring an external judge. A brief history of measuring intelligence What does it mean to measure intelligence? Alan Turing approached the problem of defining intelligence operationally, [2] replacing the question of whether a machine could think with the observable test of whether a machine’s behavior could be distinguished from that of a human. In psychometrics, the measurement problem is often approached empirically, through observing an individual’s performance across a range of tasks. In particular, an accepted phenomenon in psychometric evaluation is that performance correlates across many tasks and spans many cognitive abilities. Figure 1: Charles Spearman (Left) and Alan Turing (Right) . Image Credit: Left , Right: The New York Times This pattern was first