RoFormer: Enhanced Transformer with Rotary Position Embedding
Paper
•
2104.09864
•
Published
•
10
Papers-Fundamentals
-
-
Attention Is All You Need
Paper • 1706.03762 • Published • 44 -
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences
Paper • 2404.03715 • Published • 60 -
Zero-Shot Tokenizer Transfer
Paper • 2405.07883 • Published • 4 -
Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM
Paper • 2401.02994 • Published • 48 -
The Prompt Report: A Systematic Survey of Prompting Techniques
Paper • 2406.06608 • Published • 53 -
Extreme Compression of Large Language Models via Additive Quantization
Paper • 2401.06118 • Published • 12 -
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Paper • 2402.03300 • Published • 69 -
HyperZcdotZcdotW Operator Connects Slow-Fast Networks for Full Context Interaction
Paper • 2401.17948 • Published • 2 -
Grokfast: Accelerated Grokking by Amplifying Slow Gradients
Paper • 2405.20233 • Published • 5