Are AI systems actually smart? What does that mean? How do we measure that, as scientifically as possible?

A part of this video was brought to you by BlueDot Impact. Go to bluedot.org/sciencepetr to check out their amazing (and free!) courses on AI safety. I used to recommend these courses before they started supporting this channel.

I am so grateful to Long Phan, Dr. Mantas Mazeika, Sydney Von Arx and Joel Becker for the chance to talk to them about their research.

A massive thank you to all my friends -- please follow their channels!

Emily -- @rabbithole

Mithuna -- @LookingGlassUniverse

Toby -- @tibees

Gregor -- @veritasium

If you like what I do, and have a well paying job here is my Patreon -- https://www.patreon.com/c/SciencePetr

References here -- https://docs.google.com/document/d/1jATeVa1Ow5zSyyQu61hy90wN--EiwttUgnmqvIz4fRg/edit?usp=sharing

Benchmarks you can take --

GLUE — https://gluebenchmark.com

SuperGLUE — https://super.gluebenchmark.com

MMLU — no dedicated site; the official repo is https://github.com/hendrycks/test

GPQA (Diamond) — https://github.com/idavidrein/gpqa

HLE — https://lastexam.ai

Written directed etc etc by Petr Lebedev

Animation by Fabio Albertelli

Edited by Trenton Oliver and Axel Oliver

Petr's Shoes (not featured in the video, but endlessly supported) by I am Norda Run (their 005s are the best trail shoe I've ever run in)

I receive philanthropic funding to support these videos, but all of the opinions are always my own.

Rest in peace, Dolly Parton. The best that has ever done it.