🍡 feedmeAI
← All topics
Fine-tuning 18 items

Everything Fine-tuning

📑 arXiv 2d ago

JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models

JumpLoRA introduces adaptive sparsity in LoRA blocks via JumpReLU gating for continual learning in LLMs, achieving dynamic parameter isolation to prevent task interference. The method is modular, compatible with existing LoRA-based continual learning approaches, and significantly boosts performance over IncLoRA by constraining both magnitude and direction of updates.

📑 arXiv 2d ago

AEGIS: Anchor-Enforced Gradient Isolation for Knowledge-Preserving Vision-Language-Action Fine-Tuning

AEGIS addresses catastrophic forgetting when fine-tuning vision-language models for robotic control by preventing cross-modal gradient asymmetry—high-magnitude continuous action gradients overwriting the VLM's cross-entropy pre-trained manifold. Uses anchor-enforced gradient isolation to preserve VQA capabilities while injecting flow-matching action supervision, unlike stop-gradient or LoRA approaches.

📑 arXiv 2d ago

Stochasticity in Tokenisation Improves Robustness

Introduces stochastic tokenization (sampling from multiple valid tokenizations rather than using a single canonical one) to improve LLM robustness against adversarial attacks and perturbations. Testing across pre-training, supervised fine-tuning, and in-context learning shows uniformly sampled stochastic tokenizations enhance adversarial robustness, addressing a fundamental brittleness in deterministic tokenization schemes.

📑 arXiv 2d ago

RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration

RAGognizer uses token-level hallucination annotations from real RAG outputs as a direct training signal, integrating a detection head during fine-tuning rather than treating hallucination detection as post-hoc. The approach trains models to recognize when generated content is unsupported by retrieved context, addressing closed-domain hallucinations in retrieval-augmented generation.

💬 Reddit 4d ago

Local AI is the best

Community appreciation for local AI deployment emphasizes freedom from censorship, data harvesting, and ability to fine-tune models for personal use cases with complete privacy. Credits llama.cpp developers and open-weight model contributors for enabling on-device inference. Reflects growing preference for self-hosted solutions over cloud APIs.

🤗 Hugging Face 6d ago

Boosting Visual Instruction Tuning with Self-Supervised Guidance

MLLMs underutilize visual information during instruction tuning because many tasks can be solved with language priors alone. This method augments visual instruction tuning with self-supervised tasks (rotation prediction, color matching, cross-view correspondence) reformulated as natural language instructions. Improves fine-grained visual reasoning without increasing model size.