🟧 Hacker News 2d ago
Measuring Claude 4.7's tokenizer costs
Analysis of Claude 4.7's tokenizer efficiency and associated API costs.
Cost-optimization 3 items
Everything Cost-optimization
📑 arXiv 3d ago
Atropos: Improving Cost-Benefit Trade-off of LLM-based Agents under Self-Consistency with Early Termination and Model Hotswap
Atropos optimizes cost-benefit trade-offs for LLM agents using self-consistency by predicting when to terminate cheaper Small Language Model inference early and hotswap to larger commercial models. The system analyzes structural properties of inference paths merged into graphs to decide when local SLMs suffice versus when expensive API calls are needed.
🤗 Hugging Face 4d ago
TRACER: Trace-Based Adaptive Cost-Efficient Routing for LLM Classification
TRACER trains lightweight ML surrogates on LLM production traces to route classification traffic, activating them only when agreement with the base LLM exceeds a user-specified threshold. This approach converts logged inference data into a continuously growing training set that handles routine traffic at near-zero marginal cost while deferring edge cases to the full model.