Publications: Noam Shazeer

← Back to Author Profile

  • Affiliation: Google
  • Author ID: wsGvgA8AAAAJ
  • Cited publications: 195

Download CSV for Noam Shazeer

Title Year Citations Score
Attention is all you need
Advances in neural information processing systems 30, 2017
View on Google Scholar ↗
2017 293131 100.0%
Exploring the limits of transfer learning with a unified text-to-text transformer
Journal of machine learning research 21 (140), 1-67, 2020
View on Google Scholar ↗
2020 35958 100.0%
Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities
arXiv preprint arXiv:2507.06261, 2025
View on Google Scholar ↗
2025 5678 100.0%
Palm: Scaling language modeling with pathways
Journal of machine learning research 24 (240), 1-113, 2023
View on Google Scholar ↗
2023 10384 100.0%
Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity
Journal of Machine Learning Research 23 (120), 1-39, 2022
View on Google Scholar ↗
2022 6831 100.0%
Lamda: Language models for dialog applications
arXiv preprint arXiv:2201.08239, 2022
View on Google Scholar ↗
2022 2458 99.9%
Outrageously large neural networks: The sparsely-gated mixture-of-experts layer
arXiv preprint arXiv:1701.06538, 2017
View on Google Scholar ↗
2017 8220 99.9%
Attention is all you need, 2023
URL https://arxiv. org/abs/1706.03762 1706, 03762, 2023
View on Google Scholar ↗
2023 1714 99.8%
Gshard: Scaling giant models with conditional computation and automatic sharding
arXiv preprint arXiv:2006.16668, 2020
View on Google Scholar ↗
2020 3167 99.8%
Glu variants improve transformer
arXiv preprint arXiv:2002.05202, 2020
View on Google Scholar ↗
2020 2833 99.8%
& Polosukhin, I.(2017). Attention is all you need
Advances in neural information processing systems 30 (1), 5998-6008, 2017
View on Google Scholar ↗
2017 4673 99.8%
Image transformer
International conference on machine learning, 4055-4064, 2018
View on Google Scholar ↗
2018 2760 99.7%
Advances in neural information processing systems
Curran Associates, Inc. 30, 5998-6008, 2017
View on Google Scholar ↗
2017 3122 99.7%
Scheduled sampling for sequence prediction with recurrent neural networks
Advances in neural information processing systems 28, 2015
View on Google Scholar ↗
2015 3350 99.7%
Attention is all you need. arXiv 2017
arXiv preprint arXiv:1706.03762 10, 2017
View on Google Scholar ↗
2017 2585 99.6%
How much knowledge can you pack into the parameters of a language model?
Proceedings of the 2020 conference on empirical methods in natural language …, 2020
View on Google Scholar ↗
2020 1379 99.5%
Proceedings of the 31st international conference on neural information processing systems
Curran Associates Inc, 2017
View on Google Scholar ↗
2017 2269 99.5%
St-moe: Designing stable and transferable sparse expert models
arXiv preprint arXiv:2202.08906, 2022
View on Google Scholar ↗
2022 591 99.4%
Adafactor: Adaptive learning rates with sublinear memory cost
International conference on machine learning, 4596-4604, 2018
View on Google Scholar ↗
2018 1626 99.3%
Music transformer
arXiv preprint arXiv:1809.04281, 2018
View on Google Scholar ↗
2018 1575 99.3%
Fast transformer decoding: One write-head is all you need
arXiv preprint arXiv:1911.02150, 2019
View on Google Scholar ↗
2019 1121 99.3%
Attention is all you need
Advances in neural information processing systems 30, 2017
View on Google Scholar ↗
2017 1615 99.3%
Exploring the limits of language modeling
arXiv preprint arXiv:1602.02410, 2016
View on Google Scholar ↗
2016 1613 99.2%
Generating wikipedia by summarizing long sequences
arXiv preprint arXiv:1801.10198, 2018
View on Google Scholar ↗
2018 1238 99.1%
Lamda: Language models for dialog applications. arXiv 2022
arXiv preprint arXiv:2201.08239, 2025
View on Google Scholar ↗
2025 173 98.9%
Scaling up models and data with t5x and seqio
Journal of Machine Learning Research 24 (377), 1-8, 2023
View on Google Scholar ↗
2023 223 98.7%
End-to-end text-dependent speaker verification
2016 IEEE international conference on acoustics, speech and signal …, 2016
View on Google Scholar ↗
2016 877 98.3%
Palm: Scaling language modeling with pathways, 2022
URL https://arxiv. org/abs/2204.02311 2204, 2022
View on Google Scholar ↗
2022 236 98.3%
Tensor2tensor for neural machine translation
Proceedings of the 13th Conference of the Association for Machine …, 2018
View on Google Scholar ↗
2018 762 98.2%
Searching for efficient transformers for language modeling
Advances in neural information processing systems 34, 6010-6022, 2021
View on Google Scholar ↗
2021 292 97.8%
The best of both worlds: Combining recent advances in neural machine translation
Proceedings of the 56th annual meeting of the association for computational …, 2018
View on Google Scholar ↗
2018 606 97.7%
Mesh-tensorflow: Deep learning for supercomputers
Advances in neural information processing systems 31, 2018
View on Google Scholar ↗
2018 571 97.5%
Designing effective sparse expert models
2022 IEEE International Parallel and Distributed Processing Symposium …, 2022
View on Google Scholar ↗
2022 173 97.5%
Gspmd: general and scalable parallelization for ml computation graphs
arXiv preprint arXiv:2105.04663, 2021
View on Google Scholar ↗
2021 245 97.3%
Attention Is All You Need, August 2023
arXiv preprint arXiv:1706.03762, 16, 2023
View on Google Scholar ↗
2023 115 97.2%
Attention is all you need
Nips, 2017
View on Google Scholar ↗
2017 552 97.1%
Using large language model (s) in generating automated assistant response (s
US Patent 12,148,421, 2024
View on Google Scholar ↗
2024 92 97.1%
Blockwise parallel decoding for deep autoregressive models
Advances in Neural Information Processing Systems 31, 2018
View on Google Scholar ↗
2018 486 97.0%
Attention is all you need. CoRR abs/1706.03762 (2017)
arXiv preprint arXiv:1706.03762 1 (2), 2017
View on Google Scholar ↗
2017 477 96.7%
Attention is all you need. arXiv, 2017. doi: 10.48550
arXiv preprint arXiv.1706.03762 1706, v7, 2017
View on Google Scholar ↗
2017 471 96.6%
One model to learn them all
arXiv preprint arXiv:1706.05137, 2017
View on Google Scholar ↗
2017 452 96.4%
Renelito Delos Santos
2022
View on Google Scholar ↗
2022 118 96.0%
Do transformer modifications transfer across implementations and applications?
Proceedings of the 2021 Conference on Empirical Methods in Natural Language …, 2021
View on Google Scholar ↗
2021 156 95.2%
Gomez Aidan N., Kaiser Łukasz, Polosukhin Illia
Attention is all you need. Advances in neural information processing systems …, 2017
View on Google Scholar ↗
2017 299 94.2%
Serving content-relevant advertisements with client-side device support
US Patent 8,086,559, 2011
View on Google Scholar ↗
2011 416 93.9%
Attention is all you need. 2017. doi: 10.48550
arXiv preprint ARXIV.1706.03762 2 (2), 2017
View on Google Scholar ↗
2017 275 93.6%
Corpora generation for grammatical error correction
Proceedings of the 2019 Conference of the North American Chapter of the …, 2019
View on Google Scholar ↗
2019 191 93.1%
Hydranets: Specialized dynamic architectures for efficient inference
Proceedings of the IEEE conference on computer vision and pattern …, 2018
View on Google Scholar ↗
2018 231 92.8%
Talking-heads attention
arXiv preprint arXiv:2003.02436, 2020
View on Google Scholar ↗
2020 153 92.6%
Attention is all you need. NIPS’17
Proceedings of the 31st International Conference on Neural Information …, 2017
View on Google Scholar ↗
2017 201 91.0%
January 4-9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA
2017
View on Google Scholar ↗
2017 200 90.9%
i Illia Polosukhin
Attention is all you need, 5998-6008, 2017
View on Google Scholar ↗
2017 198 90.8%
St-moe: Designing stable and transferable sparse expert models, 2022
URL https://arxiv. org/abs/2202.08906 2, 2022
View on Google Scholar ↗
2022 57 90.5%
Fast decoding in sequence models using discrete latent variables
International Conference on Machine Learning, 2390-2399, 2018
View on Google Scholar ↗
2018 168 89.9%
Attention is all you need. arXiv: 170603762
arXiv preprint arXiv:1706.03762, 2017
View on Google Scholar ↗
2017 176 89.5%
& Le, Q.(2022). Lamda: Language models for dialog applications
arXiv preprint arXiv:2201.08239 10, 2022
View on Google Scholar ↗
2022 52 89.5%
Attention is all you need
Advances in neural information processing systems 30, 5998-6008, 2017
View on Google Scholar ↗
2017 165 88.8%
Lamda: Language models for dialog applications
2022
View on Google Scholar ↗
2022 46 88.0%
January 4-9). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA
2017
View on Google Scholar ↗
2017 148 87.4%
Attention Is All You Need.(June 2017)
arXiv preprint arXiv:1706.03762 4, 2017
View on Google Scholar ↗
2017 146 87.2%
Suggesting and/or providing targeting criteria for advertisements
US Patent 8,392,249, 2013
View on Google Scholar ↗
2013 187 86.8%
Attention-based decoder-only sequence transduction neural networks
US Patent 11,556,786, 2023
View on Google Scholar ↗
2023 26 86.6%
Attention-based sequence transduction neural networks
US Patent 10,452,978, 2019
View on Google Scholar ↗
2019 104 86.5%
Using concepts for ad targeting
US Patent App. 10/721,010, 2005
View on Google Scholar ↗
2005 236 85.7%
Attention is all you need. arXiv. org
DOI: https://doi. org/10.48550/arXiv 1706, 2017
View on Google Scholar ↗
2017 127 85.3%
Glu variants improve transformer, 2020
URL https://arxiv. org/abs, 17, 2002
View on Google Scholar ↗
2002 237 84.6%
Method and apparatus for characterizing documents based on clusters of related words
US Patent 7,383,258, 2008
View on Google Scholar ↗
2008 198 84.5%
An improved relative self-attention mechanism for transformer with application to music generation
arXiv preprint arXiv:1809.04281 2, 2018
View on Google Scholar ↗
2018 104 83.5%
Attention is all you need. arXiv e-prints, page
arXiv preprint arXiv:1706.03762 1 (2), 5, 2017
View on Google Scholar ↗
2017 100 81.3%
Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv 2017
arXiv preprint arXiv:1701.06538, 2017
View on Google Scholar ↗
2017 96 80.5%
Attention is all you need (Version 7). arXiv
arXiv preprint arXiv:1706.03762, 2017
View on Google Scholar ↗
2017 93 79.9%
Kaiser L. u., Polosukhin I., 2017
Advances in neural information processing systems 30, 5998-6008, 2017
View on Google Scholar ↗
2017 93 79.9%
31st conference on neural information processing systems (NIPS 2017)
Long Beach, CA, USA, 2017
View on Google Scholar ↗
2017 92 79.7%
Swivel: Improving embeddings by noticing what's missing
arXiv preprint arXiv:1602.02215, 2016
View on Google Scholar ↗
2016 94 78.6%
Advances in neural information processing systems. 2017
Attention is all you need, 5998-6008, 2017
View on Google Scholar ↗
2017 85 78.2%
Large scale machine learning systems and methods
US Patent 7,222,127, 2007
View on Google Scholar ↗
2007 133 77.0%
Evaluating output sequences using an auto-regressive language model neural network
US Patent 12,086,713, 2024
View on Google Scholar ↗
2024 11 76.5%
Attention is all you need, December 2017
URL http://arxiv. org/abs/1706.03762 66, 2017
View on Google Scholar ↗
2017 76 75.7%
Machine-Learned Language Models which Generate Intermediate Textual Analysis in Service of Contextual Text Generation
US Patent App. 17/749,844, 2022
View on Google Scholar ↗
2022 21 74.5%
Attention is all you need. Advances in Neural Information Processing Systems, MIT Press
2017
View on Google Scholar ↗
2017 70 73.9%
Attention is all you need. arXiv [cs. CL]
Preprint at http://arxiv. org/abs/1706.03762, 2017
View on Google Scholar ↗
2017 69 73.5%
Speech recognition with attention-based recurrent neural networks
US Patent 9,799,327, 2017
View on Google Scholar ↗
2017 69 73.5%
Attention is all you need. arXiv [Preprint](2017)
arXiv preprint arXiv:1706.03762, 2017
View on Google Scholar ↗
2017 68 73.2%
Ranking documents based on large data sets
US Patent 7,231,399, 2007
View on Google Scholar ↗
2007 110 73.1%
Attention-based image generation neural networks
US Patent 10,839,259, 2020
View on Google Scholar ↗
2020 42 72.5%
Attention is all you need. Adv Neural Inform Process Syst
2017
View on Google Scholar ↗
2017 66 72.5%
Machine-learned language models which generate intermediate textual analysis in service of contextual text generation
US Patent 11,960,848, 2024
View on Google Scholar ↗
2024 9 72.1%
A probabilistic approach to solving crossword puzzles
Artificial Intelligence 134 (1-2), 23-55, 2002
View on Google Scholar ↗
2002 110 71.1%
Music transformer: Generating music with long-term structure (2018)
arXiv preprint arXiv:1809.04281, 2018
View on Google Scholar ↗
2018 54 70.4%
Uszkoreit,“Attention is all you need,”
Advances in neural information processing systems 30, 5998-6008, 2017
View on Google Scholar ↗
2017 60 70.2%
International Conference on Machine Learning
PMLR, 2018
View on Google Scholar ↗
2018 52 69.5%
Proceedings of the 31st Conference on Neural Information Processing Systems
Curran Associates, 2017
View on Google Scholar ↗
2017 58 69.4%
Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv
arXiv preprint arXiv:1701.06538, 2017
View on Google Scholar ↗
2017 57 68.9%
The sparsely-gated mixture-of-experts layer
Outrageously large neural networks 2, 4, 2017
View on Google Scholar ↗
2017 55 68.0%
Attention is all you need. arXiv [Preprint]. doi: 10.48550
arXiv preprint arXiv.1706.03762, 2017
View on Google Scholar ↗
2017 54 67.6%
Attention is all you need. arXiv. 10.48550
arXiv preprint arXiv.1706.03762 1706, 2017
View on Google Scholar ↗
2017 54 67.6%
Multi-task multi-modal machine learning system
US Patent 10,789,427, 2020
View on Google Scholar ↗
2020 33 66.7%
Glu variants improve transformer. arXiv 2020
arXiv preprint arXiv:2002.05202 357, 2002
View on Google Scholar ↗
2002 79 64.5%
Attention is all you need study notes
Attention Is All You Need Study Notes, 2023
View on Google Scholar ↗
2023 8 64.5%
Modifying search result ranking based on implicit user feedback
US Patent 8,661,029, 2014
View on Google Scholar ↗
2014 59 64.3%
High resolution medical image analysis with spatial partitioning
arXiv preprint arXiv:1909.03108, 2019
View on Google Scholar ↗
2019 34 62.9%
Faster transformer decoding: N-gram masked self-attention
arXiv preprint arXiv:2001.04589, 2020
View on Google Scholar ↗
2020 27 61.5%
Scaling machine learning using approximate counting that uses feature hashing
US Patent 7,743,003, 2010
View on Google Scholar ↗
2010 61 61.2%
Using large language model (s) in generating automated assistant response (s)
US Patent App. 18/912,175, 2025
View on Google Scholar ↗
2025 5 61.2%
Exploring the limits of language modeling. arXiv 2016
arXiv preprint arXiv:1602.02410, 2016
View on Google Scholar ↗
2016 41 58.6%
PROVERB: The probabilistic cruciverbalist
New York Times (NYT) 792 (10), 70, 1999
View on Google Scholar ↗
1999 54 57.9%
Advances in Neural Information Processing Systems 30
NIPS, 2017
View on Google Scholar ↗
2017 35 56.5%
Attention neural networks with linear units
US Patent 12,254,411, 2025
View on Google Scholar ↗
2025 4 56.3%
Solving Crossword Puzzles as Probabilistic Constraint Satisfaction.
AAAI/IAAI, 156-162, 1999
View on Google Scholar ↗
1999 47 55.2%
Method and apparatus for learning a probabilistic generative model for text
US Patent 7,231,393, 2007
View on Google Scholar ↗
2007 46 53.9%
Attention neural networks with talking heads attention
US Patent App. 17/191,591, 2021
View on Google Scholar ↗
2021 15 53.6%
Method of spell-checking search queries
US Patent 7,194,684, 2007
View on Google Scholar ↗
2007 44 53.0%
Outrageously large neural networks: the sparsely-gated mixture-of-experts layer [OL]
2024
View on Google Scholar ↗
2024 4 52.9%
Mixture of experts neural networks
US Patent 11,790,214, 2023
View on Google Scholar ↗
2023 5 52.8%
Weakly supervised grammatical error correction using iterative decoding
arXiv preprint arXiv:1811.01710, 2018
View on Google Scholar ↗
2018 26 51.7%
Sparse non-negative matrix language modeling for skip-grams
Proc. Interspeech 2015, 1428-1432, 2015
View on Google Scholar ↗
2015 32 50.3%
Method and apparatus for learning a probabilistic generative model for text
US Patent 8,024,372, 2011
View on Google Scholar ↗
2011 36 49.7%
Image transformer
International conference on machine learning, 4055-4064, 2018
View on Google Scholar ↗
2018 23 48.5%
Liu Peter J.. 2020
Exploring the limits of transfer learning with a unified text-to-text …, 2020
View on Google Scholar ↗
2020 16 47.5%
Determining geographical relevance of web documents
US Patent 8,086,690, 2011
View on Google Scholar ↗
2011 31 46.6%
Mixture of experts neural networks
US Patent App. 16/393,063, 2019
View on Google Scholar ↗
2019 16 43.3%
Scaling machine learning using approximate counting
US Patent 8,606,730, 2013
View on Google Scholar ↗
2013 25 43.0%
Methods and apparatus for determining equivalent descriptions for an information need
US Patent 6,941,293, 2005
View on Google Scholar ↗
2005 25 42.7%
Proc. 31st International Conference on Neural Information Processing Systems (NIPS’17)
Curran Associates Inc, Red Hook, 2017
View on Google Scholar ↗
2017 20 42.7%
Attention-based sequence transduction neural networks
US Patent 10,719,764, 2020
View on Google Scholar ↗
2020 13 42.2%
Distributing tensor computations across computing devices
US Patent 10,796,225, 2020
View on Google Scholar ↗
2020 13 42.2%
Large scale machine learning systems and methods
US Patent 8,195,674, 2012
View on Google Scholar ↗
2012 24 41.3%
Speech recognition with attention-based recurrent neural networks
US Patent 9,990,918, 2018
View on Google Scholar ↗
2018 17 41.2%
Attention-based sequence transduction neural networks
US Patent App. 19/005,900, 2025
View on Google Scholar ↗
2025 2 41.0%
Attention-based sequence transduction neural networks
US Patent 12,217,173, 2025
View on Google Scholar ↗
2025 2 41.0%
Deterministic training of machine learning models
US Patent App. 18/130,339, 2023
View on Google Scholar ↗
2023 3 40.7%
Attention is all you need: Advances in neural information processing systems
NeurIPS proceedings, 2017
View on Google Scholar ↗
2017 16 37.7%
Solving crosswords with Proverb
AAAI/IAAI, 914-915, 1999
View on Google Scholar ↗
1999 16 36.7%
Neural networks with switch layers
US Patent 12,093,829, 2024
View on Google Scholar ↗
2024 2 36.0%
Attention-based sequence transduction neural networks
US Patent 11,893,483, 2024
View on Google Scholar ↗
2024 2 36.0%
Attention-based decoder-only sequence transduction neural networks
US Patent 11,886,998, 2024
View on Google Scholar ↗
2024 2 36.0%
Deterministic training of machine learning models
US Patent 12,014,276, 2024
View on Google Scholar ↗
2024 2 36.0%
Attention-based sequence transduction neural networks
US Patent App. 18/407,299, 2024
View on Google Scholar ↗
2024 2 36.0%
Skip-gram language modeling using sparse non-negative matrix probability estimation
arXiv preprint arXiv:1412.1454, 2014
View on Google Scholar ↗
2014 17 35.8%
Liu Peter J., Exploring the limits of transfer learning with a unified text-to-text transformer
arXiv preprint arXiv:1910.10683, 2019
View on Google Scholar ↗
2019 11 34.9%
Dai Andrew M
Pillai Thanumalayan Sankaranarayana, Pellat Marie, Lewkowycz Aitor, Moreira …, 2022
View on Google Scholar ↗
2022 4 32.5%
NN-grams: Unifying neural network and n-gram language models for speech recognition
arXiv preprint arXiv:1606.07470, 2016
View on Google Scholar ↗
2016 13 32.3%
Model generation for ranking documents based on large data sets
US Patent 7,743,050, 2010
View on Google Scholar ↗
2010 13 31.2%
Training recurrent neural networks to generate sequences
US Patent 10,504,023, 2019
View on Google Scholar ↗
2019 9 31.0%
Music transformer: Generating music with long-term structure
arXiv preprint, 2018
View on Google Scholar ↗
2018 10 30.3%
Constraint satisfaction with probabilistic preferences on variable values
Proc. of the 16th Nat. Conf. on AI, 1999
View on Google Scholar ↗
1999 10 30.0%
Suggesting and/or providing targeting criteria for advertisements
US Patent App. 13/784,401, 2013
View on Google Scholar ↗
2013 12 29.7%
Using concepts for ad targeting
US Patent App. 12/837,883, 2010
View on Google Scholar ↗
2010 11 28.8%
Large scale machine learning systems and methods
US Patent 7,769,763, 2010
View on Google Scholar ↗
2010 11 28.8%
Method of spell-checking search queries
US Patent 8,621,344, 2013
View on Google Scholar ↗
2013 10 27.0%
Methods and apparatus for determining equivalent descriptions for an information need
US Patent 7,392,244, 2008
View on Google Scholar ↗
2008 9 27.0%
Mixture of experts neural networks
US Patent App. 18/776,868, 2025
View on Google Scholar ↗
2025 1 26.7%
Granular neural network architecture search over low-level primitives
US Patent 12,393,840, 2025
View on Google Scholar ↗
2025 1 26.7%
Granular neural network architecture search over low-level primitives
US Patent App. 17/827,362, 2022
View on Google Scholar ↗
2022 3 26.5%
Multi-task multi-modal machine learning system
US Patent 11,494,561, 2022
View on Google Scholar ↗
2022 3 26.5%
Serving content-relevant advertisements with client-side device support
US Patent App. 12/945,999, 2011
View on Google Scholar ↗
2011 9 26.3%
Generating feature embeddings from a co-occurrence matrix
US Patent 10,685,012, 2020
View on Google Scholar ↗
2020 6 25.9%
Blockwise parallel decoding for deep autoregressive models
Advances in Neural Information Processing Systems 31, 2018
View on Google Scholar ↗
2018 7 24.6%
Sparse non-negative matrix language modeling
Transactions of the Association for Computational Linguistics 4, 329-342, 2016
View on Google Scholar ↗
2016 8 24.6%
Reranking query completions
US Patent 9,298,852, 2016
View on Google Scholar ↗
2016 8 24.6%
Ranking documents based on large data sets
US Patent 9,116,976, 2015
View on Google Scholar ↗
2015 8 24.1%
Sparse non-negative matrix language modeling for geo-annotated query session data
2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU …, 2015
View on Google Scholar ↗
2015 8 24.1%
N, GA; Ukasz, K.; Illia, P. Attention is All you Need
Proceedings of the Advances in Neural Information Processing Systems, Long …, 2017
View on Google Scholar ↗
2017 7 23.7%
Attention-based sequence transduction neural networks
US Patent 11,113,602, 2021
View on Google Scholar ↗
2021 4 22.9%
Attention-based sequence transduction neural networks
US Patent 10,956,819, 2021
View on Google Scholar ↗
2021 4 22.9%
Distributing tensor computations across computing devices
US Patent App. 17/063,034, 2021
View on Google Scholar ↗
2021 4 22.9%
Speech recognition with attention-based recurrent neural networks
US Patent 10,540,962, 2020
View on Google Scholar ↗
2020 5 22.9%
GLU variants improve transformer
arXiv preprint arXiv: 2002.05202, 2020
View on Google Scholar ↗
2020 5 22.9%
Pruning sparse non-negative matrix n-gram language models
Proceedings Interspeech 2015 2015, 1433-1437, 2015
View on Google Scholar ↗
2015 7 22.4%
Evaluating output sequences using an auto-regressive language model neural network
US Patent App. 18/797,915, 2024
View on Google Scholar ↗
2024 1 22.3%
One model to learn them all
Google Brain., 10, 2017
View on Google Scholar ↗
2017 6 21.6%
Llion Jones, Aidan N
Gomez: Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2017
View on Google Scholar ↗
2017 6 21.6%
Method of spell-checking search queries
US Patent 8,051,374, 2011
View on Google Scholar ↗
2011 6 21.5%
Identifying Object Using Generative Model
US Patent App. 12/388,245, 2010
View on Google Scholar ↗
2010 6 21.3%
Attention neural networks with conditional computation
US Patent App. 18/009,841, 2023
View on Google Scholar ↗
2023 1 19.5%
Modifying search result ranking based on implicit user feedback
US Patent 9,811,566, 2017
View on Google Scholar ↗
2017 5 19.4%
G. Aidan N., K. ukasz, and I. Polosukhin. Attention is all you need
NeurIPS, 2017
View on Google Scholar ↗
2017 5 19.4%
Kaiser Lukasz and Polosukhin Illia. Attention is all you need. In
NeurIPS, 2017, 2017
View on Google Scholar ↗
2017 5 19.4%
2017. Attention is all you need
Advances in neural information processing systems, 5998-6008, 2017
View on Google Scholar ↗
2017 5 19.4%
Method and apparatus for learning a probabilistic generative model for text
US Patent 8,412,747, 2013
View on Google Scholar ↗
2013 5 18.7%
Scaling machine learning using approximate counting
US Patent 8,019,704, 2011
View on Google Scholar ↗
2011 4 17.1%
Method and apparatus for characterizing documents based on clusters of related words
US Patent 8,688,720, 2014
View on Google Scholar ↗
2014 4 16.3%
Attention is all you need [J]. 10.48550
arXiv preprint arXiv.1706.03762, 2017
View on Google Scholar ↗
2017 3 14.3%
Large scale machine learning systems and methods
US Patent 8,364,618, 2013
View on Google Scholar ↗
2013 3 14.0%
Speech recognition with attention-based recurrent neural networks
US Patent 11,151,985, 2021
View on Google Scholar ↗
2021 2 13.8%
Attention-based image generation neural networks
US Patent 11,392,790, 2022
View on Google Scholar ↗
2022 1 11.3%
Image transformer
2018
View on Google Scholar ↗
2018 2 11.3%
Int. conf. neural info. process. syst
Attention is All You Need, 5998-6008, 2017
View on Google Scholar ↗
2017 2 11.1%
Attention Is All You Need, e-print
arXiv preprint cs.CL/1706.03762, 2017
View on Google Scholar ↗
2017 2 11.1%
N, Kaiser Łukasz, Polosukhin Illia
Advances in Neural Information Processing Systems, 5998-6008, 2017
View on Google Scholar ↗
2017 2 11.1%
Method and an apparatus to provide a personalized page
US Patent 9,141,589, 2015
View on Google Scholar ↗
2015 2 10.6%
Parallel decoding using autoregressive machine learning models
US Patent 10,521,701, 2019
View on Google Scholar ↗
2019 1 7.2%
Ranking documents based on large data sets
US Patent 10,055,461, 2018
View on Google Scholar ↗
2018 1 6.8%
Determining better ad selection, scoring, and/or presentation techniques
US Patent 9,858,590, 2018
View on Google Scholar ↗
2018 1 6.8%
Variational program inference
arXiv preprint arXiv:1006.0991, 2010
View on Google Scholar ↗
2010 1 6.6%