Everything Long-context

LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning ↗

LongAct identifies high-magnitude activations in query/key vectors during long-context processing as critical for optimization. Leverages insights from quantization and sparse reasoning structure to guide RL training for improved long-context reasoning.