AI / ML

NVFP4 inference accuracy degrades when activation outliers dominate block scales, increasing error for other values. OCGQuant addresses this by grouping outlier companions to minimize scale-induced error without adding computational overhead. This method specifically targets NVFP4 limitations, offering a more efficient alternative to rotation or mixed precision strategies.

• NVFP4 block scales are skewed by large activation outliers, hurting accuracy for other values.
• OCGQuant groups outlier companions to isolate and reduce scale-induced quantization error.
• Avoids extra compute costs from rotation or residual compensation used in other PTQ methods.
• Tailored specifically for NVFP4, improving low-bit inference precision efficiently.

This paper identifies a failure mode in multi-reward reinforcement learning where fixed scalarization weights alias qualitatively different reward profiles into a single metric. The optimization process is steered toward dimensions that are easiest or densest to optimize, causing the policy to get trapped in suboptimal states. This prevents convergence to better-balanced profiles that would improve overall task performance.

• Static aggregation weights create aliasing errors that mask distinct reward signals during LLM fine-tuning.
• Optimizers naturally drift toward easy or dense reward dimensions, ignoring harder but critical constraints.
• Policies fail to converge to balanced profiles, resulting in degraded overall task performance.
• Multi-reward RLHF systems need dynamic weighting or decomposition to avoid these aggregation-induced hacks.