LLMs

Research shows linear probes can detect corrupted context in language models with near-perfect accuracy, yet this capability does not translate into reliable failure prediction. In multi-hop arithmetic tasks, probes that identify corruption remain uninformative regarding the correctness of the final answer. Furthermore, structured confidence formats collapse to binary values with indistinguishable error rates, and probe persistence across hops fails to distinguish correct from incorrect outcomes.

• High-accuracy context corruption probes do not predict final answer correctness.
• Structured confidence scores collapse to two values with similar error rates.
• Probe persistence across reasoning hops cannot separate correct from wrong outcomes.
• Probe-based real-time monitoring faces fundamental reliability limits for error detection.

A new study reveals that current reasoning LLMs act as greedy sequential solvers when facing shared token budgets. Using an exam-style framework, researchers found that models cannot strategically divide inference resources across questions of varying difficulty and point values. This lack of strategic allocation prevents models from maximizing total scores under latency or cost constraints.

• Models treat multi-question batches as independent problems rather than optimizing for a shared global budget.
• Existing evaluations miss this flaw by testing compute allocation on single questions at a time.
• Frontier reasoning models lack the meta-cognitive ability to ration tokens based on question value.
• Greedy sequential processing leads to suboptimal performance in constrained, batched inference scenarios.

WuYuEval is a new multi-level benchmark designed to assess LLM competence in solid waste management, moving beyond general knowledge to test professional decision-making under engineering and environmental constraints. The dataset includes a Foundation Module with 4,590 closed-ended questions across eight domain categories and an Expert Module featuring 247 scenario-based open-ended questions focused on multi-objective optimization. This evaluation framework aims to measure how well models handle complex trade-offs and expert-level reasoning in this specific technical domain.

• Existing benchmarks fail to capture professional constraints in solid waste management, creating a gap in LLM assessment.
• WuYuEval tests three levels: foundational knowledge, domain reasoning, and expert decision-making.
• The Foundation Module contains 4,590 multiple-choice questions covering six task types and eight categories.
• The Expert Module uses 247 open-ended scenarios to evaluate multi-objective optimization and constraint trade-offs.
• This benchmark enables more rigorous evaluation of LLMs acting as technical assistants in engineering contexts.

Researchers introduce Prompt Embedding Probes (PEP), a white-box technique that augments standard linear probes with learnable prompt embeddings to detect hallucinations in frozen LLMs. Evaluated on Qwen3 models across TriviaQA, GSM8K, and MedQA, PEP outperforms baseline linear probes in in-distribution settings. The method also demonstrates effectiveness for pre-generation prediction and cross-model transfer, offering a practical tool for monitoring model reliability.

• PEP enhances hidden-state analysis by adding learnable prompt embeddings to linear probes.
• Tested on Qwen3 models, it improves hallucination detection over standard baselines.
• Effective for pre-generation prediction and cross-model transfer scenarios.
• Provides a white-box, non-intrusive method using frozen LLM internals.

Diffusion language models allow iterative refinement and rollback, but conventional KV caching fails because historical states change during denoising. Archer introduces a training-free method to asymmetrically cache hidden states, preserving immutable prompt contexts while allowing efficient updates to the revisable response tokens. This approach reduces inference costs by avoiding full recomputation of the global context during iterative denoising steps.

• Enables efficient rollback in diffusion LMs by handling dynamic context changes
• Training-free KV caching strategy that asymmetrically manages prompt vs. response states
• Reduces inference overhead by avoiding full recomputation during denoising updates
• Addresses the incompatibility of standard immutable KV caches with iterative refinement