Skip to content

util/hybrj: NaN/Inf handling in enorm can cause false convergence #1960

@zingale

Description

@zingale

Summary

enorm() returns 0.0 when any input component is NaN or Inf. In hybrj, fnorm == 0.0 is treated as successful convergence (info = 1). This can incorrectly classify invalid numerical states as converged solves.
 

Affected code

  • util/hybrj/hybrj_enorm.H: early return 0.0_rt for isnan/isinf
  • util/hybrj/hybrj.H: convergence check uses fnorm == 0.0_rt
     

Why this is a problem

A true zero norm and an invalid norm are semantically different. Folding both into 0.0 hides instability and can produce misleading solver status.
 

Expected behavior

Invalid numerical values should lead to a non-converged / invalid-state path, not a successful-convergence code path.
 

Suggested fix

  • Return NaN from enorm() for invalid inputs, or
  • Add an explicit validity flag / status propagation, and
  • Guard convergence checks against NaN/Inf before accepting fnorm == 0.
     

Acceptance criteria

  • Invalid residual entries no longer produce info = 1.
  • Solver reports a deterministic failure/invalid status for NaN/Inf paths.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-code-auditAI-generated code audit findings

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions