LLMs

HeadWiseKV is a training-free framework that manages KV cache memory in hybrid language models by assigning static, multilevel history windows to each physical head. This approach compresses residual global attention caches while preserving native local, recurrent, and linear pathways. By making cache demand predictable before serving, the method allocates state under an aggregate residency budget to improve inference throughput.

• Training-free compression for hybrid model global KV caches
• Static per-head windows make memory usage predictable
• Preserves local, recurrent, and linear attention paths
• Reduces GPU memory pressure during long-context decoding
• Optimizes throughput by capping aggregate cache residency

Researchers demonstrate an end-to-end post-training conversion of the Qwen3-4B model to ultra-low-bit precision, achieving 1.641 effective bits per weight. The method combines KOTMS rotation, E2M-ATQ ternarization, and GPTQ-style error compensation while keeping activations at 16-bit precision. This approach significantly reduces storage and memory bandwidth requirements without full quantization of all model components.

• Qwen3-4B conversion yields 1.641 effective bits per weight for linear parameters.
• Method uses KOTMS rotation and E2M-ATQ ternarization with GPTQ-style compensation.
• Activations remain at 16-bit precision, omitting ILA-AMP from the workflow.
• Weight-only quantization reduces storage and memory bandwidth demands.
• Study evaluates calibration sensitivity, perplexity, and deployment behavior.

GAPS introduces dimension-level conditioning to activation steering, addressing the inefficiency of applying dense vectors to all hidden dimensions regardless of relevance. By combining static separability and posterior-based gates, the method selectively intervenes only on neurons carrying concept information. This training-free approach refines the trade-off between suppressing undesired behaviors and preserving model capability.

• GAPS applies steering vectors only to specific neurons, not all dimensions.
• Uses two training-free gates: static separability and posterior selection.
• Improves behavior-capability trade-off by avoiding unnecessary interventions.
• Targets neurons with statistical concept information more precisely.

IDEEA introduces a training-free framework that steers large language models by injecting bias into activations based on input-specific clusters. Unlike prior methods that use a single fixed direction for all inputs, IDEEA matches activation clusters to find optimal steering paths for each specific query. This approach addresses the limitation that different inputs occupy distinct regions of the activation space, requiring tailored guidance toward target concepts.

• Eliminates the need for weight updates like SFT or RL for model steering.
• Adapts steering directions dynamically per input rather than using a static vector.
• Maps input activation clusters to find optimal bias injection points.
• Offers a cheaper inference-time alternative to fine-tuning for alignment tasks.