🤗 Hugging Face 4d ago
LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning
LongAct identifies high-magnitude activations in query/key vectors during long-context processing as critical for optimization. Leverages insights from quantization and sparse reasoning structure to guide RL training for improved long-context reasoning.