Publications: Ilya Sutskever

← Back to Author Profile

  • Affiliation: Co-Founder and Chief Scientist at Safe Superintelligence Inc
  • Author ID: x04W_mMAAAAJ
  • Cited publications: 156

Download CSV for Ilya Sutskever

Title Year Citations Score
Imagenet classification with deep convolutional neural networks
Advances in neural information processing systems 25, 2012
View on Google Scholar ↗
2012 176130 100.0%
Distributed representations of words and phrases and their compositionality
Advances in neural information processing systems 26, 2013
View on Google Scholar ↗
2013 47305 100.0%
Language models are few-shot learners
Advances in neural information processing systems 33, 1877-1901, 2020
View on Google Scholar ↗
2020 44327 100.0%
Learning transferable visual models from natural language supervision
International conference on machine learning, 8748-8763, 2021
View on Google Scholar ↗
2021 33206 100.0%
Language models are unsupervised multitask learners
OpenAI blog 1 (8), 9, 2019
View on Google Scholar ↗
2019 15944 100.0%
Dropout: a simple way to prevent neural networks from overfitting
The journal of machine learning research 15 (1), 1929-1958, 2014
View on Google Scholar ↗
2014 56417 100.0%
Gpt-4 technical report
arXiv preprint arXiv:2303.08774, 2023
View on Google Scholar ↗
2023 10675 100.0%
Intriguing properties of neural networks
arXiv preprint arXiv:1312.6199, 2013
View on Google Scholar ↗
2013 19269 100.0%
Improving language understanding by generative pre-training
View on Google Scholar ↗
2018 12703 100.0%
Sequence to sequence learning with neural networks
Advances in neural information processing systems 27, 2014
View on Google Scholar ↗
2014 29016 100.0%
Tensorflow: Large-scale machine learning on heterogeneous distributed systems
arXiv preprint arXiv:1603.04467, 2016
View on Google Scholar ↗
2016 59064 100.0%
Robust speech recognition via large-scale weak supervision
International conference on machine learning, 28492-28518, 2023
View on Google Scholar ↗
2023 4526 100.0%
Zero-shot text-to-image generation
International conference on machine learning, 8821-8831, 2021
View on Google Scholar ↗
2021 6080 99.9%
Mastering the game of Go with deep neural networks and tree search
nature 529 (7587), 484-489, 2016
View on Google Scholar ↗
2016 21723 99.9%
Evaluating large language models trained on code
arXiv preprint arXiv:2107.03374, 2021
View on Google Scholar ↗
2021 4474 99.9%
Improving neural networks by preventing co-adaptation of feature detectors
arXiv preprint arXiv:1207.0580, 2012
View on Google Scholar ↗
2012 12095 99.9%
Glide: Towards photorealistic image generation and editing with text-guided diffusion models
arXiv preprint arXiv:2112.10741, 2021
View on Google Scholar ↗
2021 3802 99.9%
On the importance of initialization and momentum in deep learning
International conference on machine learning, 1139-1147, 2013
View on Google Scholar ↗
2013 6849 99.9%
Infogan: Interpretable representation learning by information maximizing generative adversarial nets
Advances in neural information processing systems 29, 2016
View on Google Scholar ↗
2016 5940 99.8%
Generating long sequences with sparse transformers
arXiv preprint arXiv:1904.10509, 2019
View on Google Scholar ↗
2019 2251 99.8%
Dota 2 with large scale deep reinforcement learning
arXiv preprint arXiv:1912.06680, 2019
View on Google Scholar ↗
2019 2207 99.8%
Recurrent neural network regularization
arXiv preprint arXiv:1409.2329, 2014
View on Google Scholar ↗
2014 4020 99.7%
Consistency models
View on Google Scholar ↗
2023 991 99.7%
Generative pretraining from pixels
International conference on machine learning, 1691-1703, 2020
View on Google Scholar ↗
2020 1973 99.7%
Let's verify step by step
The Twelfth International Conference on Learning Representations, 2023
View on Google Scholar ↗
2023 869 99.7%
Deep double descent: Where bigger models and more data hurt
Journal of Statistical Mechanics: Theory and Experiment 2021 (12), 124003, 2021
View on Google Scholar ↗
2021 1206 99.6%
Gpt-4o system card
arXiv preprint arXiv:2410.21276, 2024
View on Google Scholar ↗
2024 581 99.6%
Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv 2016
arXiv preprint arXiv:1603.04467 10, 2019
View on Google Scholar ↗
2019 1362 99.5%
An empirical exploration of recurrent network architectures
International conference on machine learning, 2342-2350, 2015
View on Google Scholar ↗
2015 2547 99.5%
Improved variational inference with inverse autoregressive flow
Advances in neural information processing systems 29, 2016
View on Google Scholar ↗
2016 2293 99.5%
Evolution strategies as a scalable alternative to reinforcement learning
arXiv preprint arXiv:1703.03864, 2017
View on Google Scholar ↗
2017 1911 99.4%
Generating text with recurrent neural networks
Proceedings of the 28th international conference on machine learning (ICML …, 2011
View on Google Scholar ↗
2011 2183 99.4%
Exploiting similarities among languages for machine translation
arXiv preprint arXiv:1309.4168, 2013
View on Google Scholar ↗
2013 1971 99.3%
Jukebox: A generative model for music
arXiv preprint arXiv:2005.00341, 2020
View on Google Scholar ↗
2020 976 99.3%
Openai o1 system card
arXiv preprint arXiv:2412.16720, 2024
View on Google Scholar ↗
2024 260 99.1%
Continuous deep q-learning with model-based acceleration
International conference on machine learning, 2829-2838, 2016
View on Google Scholar ↗
2016 1370 99.1%
Improving language understanding by generative pre-training.(2018)
View on Google Scholar ↗
2018 1206 99.0%
RL: Fast Reinforcement Learning via Slow Reinforcement Learning
arXiv preprint arXiv:1611.02779, 2016
View on Google Scholar ↗
2016 1269 99.0%
Language models can explain neurons in language models
URL https://openaipublic. blob. core. windows. net/neuron-explainer/paper …, 2023
View on Google Scholar ↗
2023 268 99.0%
Weak-to-strong generalization: Eliciting strong capabilities with weak supervision
arXiv preprint arXiv:2312.09390, 2023
View on Google Scholar ↗
2023 252 98.9%
Grammar as a foreign language
Advances in neural information processing systems 28, 2015
View on Google Scholar ↗
2015 1162 98.8%
Ffjord: Free-form continuous dynamics for scalable reversible generative models
arXiv preprint arXiv:1810.01367, 2018
View on Google Scholar ↗
2018 999 98.8%
Addressing the rare word problem in neural machine translation
arXiv preprint arXiv:1410.8206, 2014
View on Google Scholar ↗
2014 1015 98.6%
Multi-task sequence to sequence learning
arXiv preprint arXiv:1511.06114, 2015
View on Google Scholar ↗
2015 981 98.6%
Improving language understanding with unsupervised learning
OpenAI Res, 2018
View on Google Scholar ↗
2018 826 98.4%
One-shot imitation learning
Advances in neural information processing systems 30, 2017
View on Google Scholar ↗
2017 855 98.4%
Variational lossy autoencoder
arXiv preprint arXiv:1611.02731, 2016
View on Google Scholar ↗
2016 851 98.2%
Scaling and evaluating sparse autoencoders
arXiv preprint arXiv:2406.04093, 2024
View on Google Scholar ↗
2024 136 98.1%
Learning to generate reviews and discovering sentiment
View on Google Scholar ↗
2018 620 97.7%
Training recurrent neural networks
View on Google Scholar ↗
2013 746 97.6%
Learning recurrent neural networks with hessian-free optimization
Proceedings of the 28th international conference on machine learning (ICML …, 2011
View on Google Scholar ↗
2011 848 97.5%
TensorFlow: Large-scale machine learning on heterogeneous systems (2015), software available from tensorflow. org
View on Google Scholar ↗
2019 449 97.5%
Learning to execute
arXiv preprint arXiv:1410.4615, 2014
View on Google Scholar ↗
2014 677 97.5%
Adding gradient noise improves learning for very deep networks
arXiv preprint arXiv:1511.06807, 2015
View on Google Scholar ↗
2015 640 97.5%
Formal mathematics statement curriculum learning
arXiv preprint arXiv:2202.01344, 2022
View on Google Scholar ↗
2022 173 97.5%
Gradient-based learning applied to document recognition
Commun. Acm 60, 84-90, 2017
View on Google Scholar ↗
2017 596 97.4%
AI and Compute
View on Google Scholar ↗
2018 529 97.3%
Language models are few-shot learners
arXiv preprint arXiv:2005.14165 1, 3, 2020
View on Google Scholar ↗
2020 346 97.3%
Emergent complexity via multi-agent competition
arXiv preprint arXiv:1710.03748, 2017
View on Google Scholar ↗
2017 521 96.9%
TensorFlow: large-scale machine learning on heterogeneous distributed systems (2016)
arXiv preprint arXiv:1603.04467 52, 2015
View on Google Scholar ↗
2015 557 96.9%
Generative language modeling for automated theorem proving
arXiv preprint arXiv:2009.03393, 2020
View on Google Scholar ↗
2020 286 96.5%
Continuous adaptation via meta-learning in nonstationary and competitive environments
arXiv preprint arXiv:1710.03641, 2017
View on Google Scholar ↗
2017 447 96.4%
Better language models and their implications
OpenAI blog 1 (2), 2019
View on Google Scholar ↗
2019 303 96.0%
Language Models are Few-Shot Learners, arxiv, 2020
View on Google Scholar ↗
2005 690 96.0%
GE Hinton Imagenet classification with deep convolutional neural networks., 2012, 25
DOI: https://doi. org/10.1145/3065386, 1097-1105, 2024
View on Google Scholar ↗
2024 66 95.8%
Neural gpus learn algorithms
arXiv preprint arXiv:1511.08228, 2015
View on Google Scholar ↗
2015 429 95.7%
The recurrent temporal restricted boltzmann machine
Advances in neural information processing systems 21, 2008
View on Google Scholar ↗
2008 618 95.7%
Learning factored representations in a deep mixture of experts
arXiv preprint arXiv:1312.4314, 2013
View on Google Scholar ↗
2013 404 94.7%
Third-person imitation learning
arXiv preprint arXiv:1703.01703, 2017
View on Google Scholar ↗
2017 280 93.7%
Reinforcement learning neural turing machines-revised
arXiv preprint arXiv:1505.00521, 2015
View on Google Scholar ↗
2015 309 93.5%
DALL· E: Creating images from text
OpenAI blog 2, 2021
View on Google Scholar ↗
2021 120 93.4%
Governance of superintelligence
OpenAI, May 22, 2023
View on Google Scholar ↗
2023 51 93.3%
Neural programmer: Inducing latent programs with gradient descent
arXiv preprint arXiv:1511.04834, 2015
View on Google Scholar ↗
2015 295 93.2%
Improving language understanding by generative pre-training. OpenAI
Preprint, 1-12, 2018
View on Google Scholar ↗
2018 226 92.6%
Scalable reversible generative models with free-form continuous dynamics
International Conference on Learning Representations 3, 2019
View on Google Scholar ↗
2019 161 91.6%
Imagenet classification with deep convolutional neural networks, 2012
Advances in neural information processing systems 25 (13), 770-778, 2019
View on Google Scholar ↗
2019 155 91.2%
Training deep and recurrent networks with hessian-free optimization
Neural Networks: Tricks of the Trade: Second Edition, 479-535, 2012
View on Google Scholar ↗
2012 276 91.0%
Some considerations on learning to explore via meta-reinforcement learning
arXiv preprint arXiv:1803.01118, 2018
View on Google Scholar ↗
2018 183 90.8%
Modelling relational data using bayesian clustered tensor factorization
Advances in neural information processing systems 22, 2009
View on Google Scholar ↗
2009 312 90.8%
Thore Graepel, Thore and Demis Hassabis
Mastering the game of Go with deep neural networks and tree search.” nature …, 2016
View on Google Scholar ↗
2016 200 90.3%
Subword language modeling with neural networks
preprint (http://www. fit. vutbr. cz/imikolov/rnnlm/char. pdf) 8 (67), 2012
View on Google Scholar ↗
2012 247 89.7%
Learning multilevel distributed representations for high-dimensional sequences
Artificial intelligence and statistics, 548-555, 2007
View on Google Scholar ↗
2007 293 89.4%
Dota 2 with large scale deep reinforcement learning
arXiv preprint arXiv:1912.06680 2, 2019
View on Google Scholar ↗
2019 127 89.1%
Language Models are Few-Shot Learners. 2020. doi: 10.48550
arxiv 1, 2005
View on Google Scholar ↗
2005 305 89.1%
Neural random-access machines
arXiv preprint arXiv:1511.06392, 2015
View on Google Scholar ↗
2015 192 88.8%
TensorFlow: Large-scale machine learning on heterogeneous systems. arXiv 2015
arXiv preprint arXiv:1603.04467, 2016
View on Google Scholar ↗
2016 170 88.4%
Recurrent neural network regularization. arXiv 2014
arXiv preprint arXiv:1409.2329, 2014
View on Google Scholar ↗
2014 195 88.4%
Move evaluation in Go using deep convolutional neural networks
arXiv preprint arXiv:1412.6564, 2014
View on Google Scholar ↗
2014 192 88.2%
Muprop: Unbiased backpropagation for stochastic neural networks
arXiv preprint arXiv:1511.05176, 2015
View on Google Scholar ↗
2015 179 87.9%
Learning transferable visual models from natural language supervision. arXiv 2021
arXiv preprint arXiv:2103.00020 10, 2021
View on Google Scholar ↗
2021 68 87.4%
ImageNet classification with deep convolutional neural networks
Advances in neural information processing systems 25, 1097-1105, 2012
View on Google Scholar ↗
2012 200 87.0%
Gamepad: A learning environment for theorem proving
arXiv preprint arXiv:1806.00608, 2018
View on Google Scholar ↗
2018 124 86.1%
Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv 2021
arXiv preprint arXiv:2112.10741, 2021
View on Google Scholar ↗
2021 60 85.5%
An online sequence-to-sequence model using partial conditioning
Advances in neural information processing systems 29, 2016
View on Google Scholar ↗
2016 138 85.5%
Tensorflow: Large-scale machine learning on heterogeneous distributed systems
arXiv preprint arXiv:1603.04467, 2016
View on Google Scholar ↗
2016 132 84.7%
Deep, narrow sigmoid belief networks are universal approximators
Neural computation 20 (11), 2629-2636, 2008
View on Google Scholar ↗
2008 197 84.4%
Recurrent neural network regularization. arXiv
arXiv preprint arXiv:1409.2329 8, 2014
View on Google Scholar ↗
2014 147 84.3%
Language models are unsupervised multitask learners. OpenAI blog (2019)
URL: https://d4mucfpksywv. cloudfront. net/better-language-models/language …, 2019
View on Google Scholar ↗
2019 90 84.3%
Sequence to sequence learning with neural networks. arXiv 2014
arXiv preprint arXiv:1409.3215, 2014
View on Google Scholar ↗
2014 141 83.6%
Distribution augmentation for generative modeling
International Conference on Machine Learning, 5006-5019, 2020
View on Google Scholar ↗
2020 64 81.3%
On the convergence properties of contrastive divergence
Proceedings of the thirteenth international conference on artificial …, 2010
View on Google Scholar ↗
2010 159 81.2%
Visualizing similarity data with a mixture of maps
Artificial intelligence and statistics, 67-74, 2007
View on Google Scholar ↗
2007 156 79.9%
AI and compute, 2018
URL https://openai. com/blog/ai-and-compute 4, 2018
View on Google Scholar ↗
2018 81 79.0%
Distributed representations of words and phrases and their compositionality. arXiv 2013
arXiv preprint arXiv:1310.4546, 2013
View on Google Scholar ↗
2013 113 78.3%
TensorFlow
Large-scale machine learning on heterogeneous systems 11, 2015
View on Google Scholar ↗
2015 92 76.5%
Language models are few-shot learners. CoRR abs/2005.14165 (2020)
URL: https://arxiv. org/abs/2005.14165, 2005
View on Google Scholar ↗
2005 123 74.5%
& Zheng
X.: Tensorflow: Large-scale machine learning on heterogeneous distributed …, 2016
View on Google Scholar ↗
2016 75 73.7%
Generating long sequences with sparse transformers.(2019)
arXiv preprint arXiv:1904.10509, 2019
View on Google Scholar ↗
2019 52 73.5%
Unsupervised neural machine translation with generative language models only
arXiv preprint arXiv:2110.05448, 2021
View on Google Scholar ↗
2021 32 73.4%
Estimating the hessian by back-propagating curvature
arXiv preprint arXiv:1206.6464, 2012
View on Google Scholar ↗
2012 92 72.6%
Towards principled unsupervised learning
arXiv preprint arXiv:1511.06440, 2015
View on Google Scholar ↗
2015 73 71.2%
Tensorflow: Large-scale machine learning on heterogeneous distributed systems. CoRR abs/1603.04467 (2016)
arXiv preprint arXiv:1603.04467, 2016
View on Google Scholar ↗
2016 63 69.6%
Proceedings of the 26th international conference on neural information processing systems
ACM, EEUU 2, 3111-9, 2013
View on Google Scholar ↗
2013 75 69.3%
Evolution strategies as a scalable alternative to reinforcement learning (2017)
arXiv preprint arXiv:1703.03864, 476-485, 2017
View on Google Scholar ↗
2017 55 68.0%
Statement on AI risk
Center for AI safety 31 (5), 2023, 2023
View on Google Scholar ↗
2023 9 67.2%
Learning online alignments with continuous rewards policy gradient
2017 IEEE International Conference on Acoustics, Speech and Signal …, 2017
View on Google Scholar ↗
2017 53 67.1%
2012 AlexNet
Adv. Neural Inf. Process. Syst, 1-9, 2012
View on Google Scholar ↗
2012 69 65.9%
Cnn 实际训练的
Commun. ACM 60 (6), 84-90, 2012
View on Google Scholar ↗
2012 68 65.5%
TensorFlow: large-scale machine learning on heterogeneous systems, Software available from tensorflow. org
arXiv preprint arXiv:1603.04467, 2015
View on Google Scholar ↗
2015 57 65.0%
Intriguing properties of neural networks. arXiv preprint (2013)
arXiv preprint arXiv:1312.6199, 2013
View on Google Scholar ↗
2013 61 64.3%
Stochastic k-neighborhood selection for supervised and unsupervised learning
International Conference on Machine Learning, 199-207, 2013
View on Google Scholar ↗
2013 60 63.9%
International conference on machine learning
On the importance of initialization and momentum in deep learning, 1139-1147, 2013
View on Google Scholar ↗
2013 59 63.5%
TensorFlow: Large-scale machine learning on heterogeneous systems. tensorflow. org
Python library, 2015
View on Google Scholar ↗
2015 50 61.7%
Mimicking go experts with convolutional neural networks
Artificial Neural Networks-ICANN 2008: 18th International Conference, Prague …, 2008
View on Google Scholar ↗
2008 61 60.3%
Temporal-kernel recurrent neural networks
Neural Networks 23 (2), 239-243, 2010
View on Google Scholar ↗
2010 58 60.0%
A neural transducer
arXiv preprint arXiv:1511.04868, 2015
View on Google Scholar ↗
2015 46 59.5%
Salakhutdinov
Improving neural networks by preventing co-adaptation of feature detectors, 2012
View on Google Scholar ↗
2012 50 57.9%
Cardinality restricted boltzmann machines
Advances in neural information processing systems 25, 2012
View on Google Scholar ↗
2012 44 54.9%
ImageNet classification with deep convolutional neural networks (AlexNet) ImageNet classification with deep convolutional neural networks (AlexNet)
Actorsfit. Com, 2023
View on Google Scholar ↗
2023 5 52.8%
Data normalization in the learning of restricted Boltzmann machines
Department of Computer Science, University of Toronto, Technical Report UTML …, 2011
View on Google Scholar ↗
2011 33 48.0%
Parallelizable sampling of markov random fields
Proceedings of the Thirteenth International Conference on Artificial …, 2010
View on Google Scholar ↗
2010 31 46.4%
Using matrices to model symbolic relationship
Advances in neural information processing systems 21, 2008
View on Google Scholar ↗
2008 30 45.6%
Extensions and limitations of the neural gpu
arXiv preprint arXiv:1611.00736, 2016
View on Google Scholar ↗
2016 23 43.9%
Introducing OpenAI
Zugang: https://blog. openai. com/introducing-openai/(15.12. 2019) and the …, 2015
View on Google Scholar ↗
2015 23 42.5%
Language models are unsupervised multitask learners. OpenAI
View on Google Scholar ↗
2013 23 41.3%
Developing deep learning applications for life science and pharma industry
Drug research 68 (06), 305-310, 2018
View on Google Scholar ↗
2018 15 38.4%
Martin wa enberg, martin wicke, yuan yu, and xiaoqiang zheng. 2015. tensorflow: Large-scale machine learning on heterogeneous systems.(2015). hp
tensor ow. org/So ware available from tensor ow. org, 2015
View on Google Scholar ↗
2015 19 38.3%
Systems and methods for generating natural language using language models trained on computer code
US Patent 12,008,341, 2024
View on Google Scholar ↗
2024 2 36.0%
Sequence to sequence learning with neural networks. NIPS
‘, 2014
View on Google Scholar ↗
2014 17 35.8%
ImageNet classification with deep convolutional neural networks. 2012: 1097–1105
Last accessed Oct 1, 2019
View on Google Scholar ↗
2019 11 34.9%
A simpler unified analysis of budget perceptrons
Proceedings of the 26th Annual International Conference on Machine Learning …, 2009
View on Google Scholar ↗
2009 14 33.2%
An online sequence-to-sequence model for noisy speech recognition
arXiv preprint arXiv:1706.06428, 2017
View on Google Scholar ↗
2017 8 25.5%
Murray
Dere k G., Steiner, Benoit, Tucker, Paul, Vasudevan, Vijay, Warden, Pete …, 2016
View on Google Scholar ↗
2016 8 24.6%
In Z. Ghahramani, M. Welling, C. Cortes, ND Lawrence, K., & Q. Weinberger
Advances in neural information processing systems 27, 2014
View on Google Scholar ↗
2014 7 22.3%
Systems and methods for generating code using language models trained on computer code
US Patent 12,061,880, 2024
View on Google Scholar ↗
2024 1 22.3%
System and method for parallelizing convolutional neural networks
US Patent 11,928,577, 2024
View on Google Scholar ↗
2024 1 22.3%
Multi-task automatic speech recognition system
US Patent 12,079,587, 2024
View on Google Scholar ↗
2024 1 22.3%
Proceedings of the 28th International Conference on Machine Learning (ICML-11)
Citeseer, 2011
View on Google Scholar ↗
2011 6 21.5%
Improving neural networks by preventing co-adaptation of feature detectors. arXiv: Neural and Evolutionary Computing
View on Google Scholar ↗
2012 5 18.8%
Grammar as a foreign language
Advances in Neural Information Processing Systems, 2015
View on Google Scholar ↗
2015 5 18.6%
Distributed Representations of Words and Phrases and their Compositionality. NIPS’13: Proceedings of the 26th International Conference on Neural Information Processing Systems 2: 3111–3119 0. Mohan D, Nair LR (2021) Probit Regressive Tversky Indexed Rocchio Convolutive Deep Neural Learning for Legal Document Data Analytics
International Journal of Intelligent Systems and Applications in Engineering …, 2013
View on Google Scholar ↗
2013 4 16.5%
Distributed Representations of Words and Phrases and Their Compositionality. arXiv
URL https://arxiv. org/abs/1310.4546, 2015
View on Google Scholar ↗
2015 4 16.3%
From Photographic Image to Computer Vision
Cambridge University Press, 2019
View on Google Scholar ↗
2019 2 12.0%
G, Corrado, & J
Dean,“Distributed Representations of Words and Phrases and their …, 2013
View on Google Scholar ↗
2013 2 10.9%
International Conference on Neural Information Processing Systems
Curran Associates Inc., 2012
View on Google Scholar ↗
2012 2 10.7%
Tensorflow: large-scale machine learning on heterogeneous distributed systems. CoRR [Online], abs/1603.04467
View on Google Scholar ↗
2016 2 10.6%