LLMs

CAST optimizes speculative LLM inference by organizing multiple high-scoring token candidates from block drafters into a verification tree. Instead of discarding non-top chains, it verifies them in a single target model pass, leveraging already-computed drafts. The method dynamically adjusts tree width by adding candidates only when the expected performance gain exceeds the computational cost.

• Reuses existing block drafter scores by verifying multiple candidates instead of just the top one.
• Reduces target model passes by packing candidates into a single verification tree structure.
• Requires no changes to target model weights, drafter architecture, or decoding rules.
• Dynamically scales tree width based on cost-benefit analysis of additional candidates.

Researchers evaluated PRM-Pruned Fragment Grafting (PPFG) to mitigate diversity collapse in parallel chain-of-thought reasoning. Testing on Qwen2.5-7B-Instruct with Math-Shepherd across the full MATH500 dataset, the method failed to outperform independent decoding. The findings indicate that PPFG alone is insufficient for performance gains, suggesting prior successes relied on additional compensating mechanisms.

• PPFG extracts high-PRM prefixes from pruned chains to graft into decoding siblings.
• No statistical improvement observed on Qwen2.5-7B-Instruct with Math-Shepherd on MATH500.
• Stagnation- and random-targeting variants both showed inert results.
• Prior fragment-grafting gains likely required extra ingredients beyond simple grafting.
• Cross-trajectory step-level transfer is not automatically cost-effective without tuning.

Reading verdicts from the first token's logits is a common cheap evaluation method, but it systematically inflates position bias by treating initial tokens as final decisions. For several Qwen3 models, 12% to 49% of pairs fail to lead with a verdict, causing forced reads to incorrectly favor the first response shown. This distortion acts as an upper bound on reported bias metrics, misleading researchers about actual judge behavior.

• First-token logit reading inflates position bias metrics, acting as an upper bound.
• Qwen3 judges skip verdict tokens in 12-49% of cases, skewing forced-read results.
• Forced reads incorrectly select the first response when no verdict token is present.
• Llama-3.1-8B and Phi-3.5-mini show minimal skipping (<3%), reducing this error.
• Evaluation harnesses using constrained decoding may produce misleading bias figures.

A study on role-specialized QA pipelines reveals that passing rationales from a reasoner to a verifier yields minimal gains in answer accuracy compared to providing no rationale at all. Using DeepSeek models on multi-hop datasets, the researchers found that corrupted rationales significantly shift a verifier's support judgments by up to 22%, while harmless paraphrases cause negligible changes. This suggests rationales may create a new failure surface rather than reliably improving answer quality.

• Faithful rationales provide almost no accuracy boost over no rationale in role-specialized QA pipelines.
• Corrupted rationales strongly distort verifier support judgments, shifting them by 10-22%.
• Harmless paraphrases of rationales have negligible impact (0-2.5%) on support assessments.
• Designers should view rationales as a potential failure surface, not a guaranteed quality signal.

Research demonstrates that fine-tuning with minimal harmful examples can strip aligned LLMs of their refusal capabilities. However, safety behaviors remain linearly separable from benign ones even after the attack. By identifying the specific depth where refusal localizes, defenders can patch compromised models by restoring clean hidden states or freezing layers up to that transition point.

• Few-shot harmful fine-tuning effectively disables model refusal across multiple model families.
• Safety and benign prompts remain linearly separable in hidden states post-attack.
• Patching clean hidden states at the transition depth restores refusal behavior reproducibly.
• Freezing layers up to the identified safety depth offers a robust defense against adaptation.
• Localizing safety to specific layers enables targeted recovery without full retraining.