Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja, Richard Ren, Ryan Kim, J. Hausenloy, Oliver Zhang, Mantas Mazeika, Daron Anderson, Tung Nguyen, Imad Ali Shah, Mobeen Mahmood, Fiona Feng, Steven Y. Feng, Haoran Zhao, Michael Yu, Varun Prashant Gangal, Chelsea Zou, Zihan Wang, Jessica Wang, Pawan Kumar, Oleksandr Pokutnyi, Robert Gerbicz, S. Popov, John-Clark Levin, Mstyslav Kazakov, Johannes Schmitt, Geoff Galgon, Á. Sánchez, Yongki Lee, Will Yeadon, Scott C. Sauers, Marc Roth, C. Agu, Soren Riis, Fabian Giska, Saiteja Utpala, Zachary Giboney, G. M. Goshu, J. Xavier, Sarah-Jane Crowson, M. Naiya, Noah Burns, Lennart Finke, Zerui Cheng, Hyunwoo Park, Francesco Fournier-Facio, John B. Wydallis, M. Nandor, Ankit Singh, Tim Gehrunger, Jiaqi Cai, Ben McCarty, D. Duclosel, Jungbae Nam, Jennifer A. Zampese, R. Hoerr, Aras Bacho, Gautier Abou Loume, A. Galal, Hangrui Cao, Alexis C Garretson, Damien Sileo, Qiuyu Ren, Doru Cojoc, Pavel Arkhipov, Usman Qazi, Lianghui Li, S. Motwani, Christian Schröder de Witt, Edwin Taylor, Johannes Veith, Eric Singer, Taylor D. Hartman, P. Rissone, Jaehyeok Jin, J. Shi, Chris G. Willcocks, Joshua Robinson, Aleksandar Mikov, Ameya Prabhu, Longke Tang, Xavier Alapont, Justine Leon Uro, Kevin Zhou, Emily de Oliveira Santos, Andrey Pupasov Maksimov, Edward Vendrow, Kengo Zenitani, Julien Guillod, Yuqi Li, Joshua Vendrow, Vladyslav M. Kuchkin, Ze-An Ng, Pierre Marion, Denis Efremov, Jayson Lynch, Kaiqu Liang, Andrew Gritsevskiy, Dakotah Martinez, B. Pageler, Nick Crispino, D. Zvonkine, Natanael Wildner Fraga, Saeed Soori, O. Press, Henry Tang, Julian Salazar, Sean R. Green, Lina Brussel, Moon Twayana, Aymeric Dieuleveut, T. Rogers, Wenjin Zhang, Bikun Li, Jinzhou Yang, Arun Rao, Gabriel Loiseau, Mikhail Kalinin, Marco Lukas, Ciprian Manolescu, Subrata Mishra, A. Kamdoum, T. Kreiman, Tad Hogg, Alvin Jin, Carlo Bosio, Gongbo Sun, Brian P. Coppola, Tim Tarver, H. Heidinger, Rafael Sayous, Stefan Ivanov, Joe Cavanagh, Jiawei Shen, Joseph Marvin Imperial, P. Schwaller, Shaipranesh Senthilkuma, Andrés M Bran, A. Dehghan, A. Algaba, Brecht Verbeken, David A. Noever, V. RagavendranP, Lisa Schut, Ilia Sucholutsky, Evgenii Zheltonozhskii, Derek Lim, Richárd Stanley, Shankar Sivarajan, Tong Yang, John Maar, Julian Wykowski, Martí Oller, Jennifer Sandlin, Anmol Sahu, Yuzheng Hu, Sara Fish, N. Heydari, Archimedes T. Apronti, Kai Rawal, Tobías García Vilchis, Yuexuan Zu, M. Lackner, James Koppel, Jeremy Nguyen, D. Antonenko, Steffi Chern, Bingchen Zhao, Pierrot Arsene, A. Goldfarb, Sergey Ivanov, Rafal Poswiata, Chenguang Wang, Daofeng Li, Donato Crisostomi, Donato Crisostomi, Benjamin Myklebust, Archan Sen, D. Perrella, Nurdin Kaparov, Mark H Inlow, A. Zang, Elliott Thornley, D. Orel, Vladislav Poritski, S. Ben-David, Zachary Berger, Parker Whitfill, Michael Foster, D. Munro, Linh Ho, Dan Bar Hava, Aleksey Kuchkin, Robert Lauff, David Holmes, F. Sommerhage, K. Schneider, Z. Kazibwe, Nate Stambaugh, Mukhwinder Singh, I. Magoulas, D. Clarke, D. Kim, F. Dias, Dae Hyun Kim, K. Agarwal, Victor Efren Guadarrama Vilchis, Immo Klose, Christoph Demian, Ujjwala Anantheswaran, Adam Zweiger, Guglielmo Albani, Jeffery Li, Nicolas Daans, M. Radionov, Václav Rozhoň, Ziqiao Ma, Christian Stump, Mohammed Berkani, Jacob Platnick, Volodymyr Nevirkovets, Luke Basler, M. Piccardo, F. Jeanplong, Niv Cohen, Josef Tkadlec, P. Rosu, Piotr Padlewski, Stanislaw Barzowski, Kyle Montgomery, Aline Menezes, Arkil Patel, Zixuan Wang, Jamie Tucker-Foltz, Jack Stade, Tom Goertzen, F. Kazemi, Jeremiah Milbauer, John Arnold Ambay, Jeremiah Milbauer, Yan Carlos Leyva Labrador, Alan Givr'e, Hew Wolff, V. Rossbach, M. Aziz, Younesse Kaddar, Yanxu Chen, Robin Zhang, Jiayi Pan, Antonio Terpin, Jiayi Pan, Hailey Schoelkopf, E. Zheng, Avishy Carmi, Adam Jones, Jainam Shah, Ethan D. L. Brown, Kelin Zhu, M. Bartolo, Richard S. Wheeler, Andrew Ho, Shaul Barkan, Jiaqi Wang, Martin Stehberger, Egor Kretov, Kaustubh Sridhar, Kaustubh Sridhar, Anji Zhang, D. Pyda, Joanna Tam, D. Cunningham, V. Goryachev, Demosthenes Patramanis, Michael Krause, Andrew Redenti, Daniel Bugas, David Aldous, Jesyin Lai, Shannon Coleman, Mohsen Bahaloo, Jiangnan Xu, Sangwon Lee, Sandy Zhao, Ning Tang, Michael K. Cohen, Micah Carroll, Micah Carroll, J. Kirchner, Stefan Steinerberger, M. Ovchynnikov, Jason O. Matos, Adithya Shenoy, Benedito Alves de Oliveira Junior, Michael Wang, Yuzhou Nie, Paolo Giordano, Philipp Petersen, Anna Sztyber-Betley, Priti Shukla, J. Crozier, Antonella Pinto, Shreyas Verma, Prashant Joshi, Zheng-Xin Yong, Allison Tee, Zheng-Xin Yong, Orion Weller, Raghav Singhal, Orion Weller, A. Ivanov, Seri Khoury, H. Mostaghimi, Kunvar Thaman, Qijia Chen, Tran Quoc Khánh, Jacob Loader, Stefano Cavalleri, Hannah Szlyk, Zachary Brown, Jonathan Roberts, William Alley, Kunyang Sun, Ryan Stendall, Max Lamparth, Anka Reuel, Ting Wang, Hanmeng Xu, Sreenivas Goud Raparthi, Pablo Hern'andez-C'amara, Freddie Martin, Dmitry Malishev, T. Preu, Tomasz Korbak, Marcus Abramovitch, Dominic J. Williamson, Ziye Chen, Bir'o B'alint, M Saiful Bari, P. Kassani, Zihao Wang, B. Ansarinejad, Laxman Prasad Goswami, Yewen Sun, Hossam Elgnainy, D. Tordera, George Balabanian, Earth Anderson, Lynna Kvistad, Alejandro Jos'e Moyano, Rajat Maheshwari, Ahmad Sakor, Murat Eron, I. McAlister, Javier Gimenez, Innocent Enyekwe, Andrew Favre D.O., Shailesh Shah, Xiaoxiang Zhou, Firuz Kamalov, Ronald Clark, Sherwin Abdoli, Tim Santens, Khalida Meer, Harrison K Wang, K. Ramakrishnan, Evan Chen, A. Tomasiello, G. Luca, Shi-Zhuo Looi, Vinh-Kha Le, Noam Kolt, Niels Mundler, Avi Semler, Niels Mündler, Jacob Drori, Carl J Fossum, Milind Jagota, Ronak Pradeep, Honglu Fan, Tej Shah, Jonathan Eicher, Michael Chen, Kushal Thaman, William Merrill, Carter Harris, Jason Gross, Ilya Gusev, Asankhaya Sharma, Shashank Agnihotri, P. Zhelnov, Siranut Usawasutsakorn, Mohammadreza Mofayezi, Sergei Bogdanov, A. Piperski, Marc Carauleanu, David K. Zhang, Dylan Ler, Roman Leventov, Ignat Soroko, Thorben Jansen, Pascal Lauer, Joshua Duersch, Vage Taamazyan, Wiktor Morak, Wenjie Ma, William Held, T. Huy, Ruicheng Xian, A. Zebaze, Mohanad Mohamed, Julian Noah Leser, Michelle Yuan, Laila Yacar, Johannes Lengler, Hossein Shahrtash, Edson Oliveira, Joseph W Jackson, D. Gonzalez, Andy Zou, Muthu Chidambaram, Timothy Manik, Hector Haffenden, Dashiell Stander, A. Dasouqi, Alexander Shen, Emilien Duc, Bita Golshani, David Stap, Mikalai Uzhou, Alina B. Zhidkovskaya, Lukas Lewark, M'aty'as Vincze, Dustin Wehr, Colin Tang, Zaki Hossain, Shaun Phillips, M. Jiang, Fredrik Ekstrom, Angela Hammon, Oam Patel, Nicolas Remy, Faraz Farhidi, G. Medley, F. Mohammadzadeh, Madellene Peñaflor, Haile Kassahun, Alena Friedrich, Claire Sparrow, Taom Sakal, Omkar Dhamane, Ali Khajegili Mirabadi, E. Hallman, M. Battaglia, Mohammad Maghsoudimehrabani, Hieu Hoang, Alon Amit, D. Hulbert, R. Pereira, S. Weber, Stephen Mensah, Nathan Andre, Anton Peristyy, Chris Harjadi, Himanshu Gupta, S. Malina, Himanshu Gupta, Will Cai, Mustafa Mehkary, Frank Reidegeld, Anna-Katharina Dick, Cary Friday, Jasdeep Sidhu, Wanyoung Kim, M. Costa, Hubeyb Gurdogan, Brian Weber, Harsh Kumar, Tong Jiang, Arunim Agarwal, Chiara Ceconello, W. S. Vaz, Chao Zhuang, Haon Park, A. Tawfeek, Haon Park, Michael Kirchhof, Linjie Dai, Michael Kirchhof, Johan Ferret, Yuzhou Wang
Abstract
Capability benchmark testing AI systems