Skip to content

Handle empty inputs in XLA _linalg_svd#3252

Open
IvanYashchuk wants to merge 9 commits into
pytorch:masterfrom
IvanYashchuk:svd
Open

Handle empty inputs in XLA _linalg_svd#3252
IvanYashchuk wants to merge 9 commits into
pytorch:masterfrom
IvanYashchuk:svd

Conversation

@IvanYashchuk
Copy link
Copy Markdown
Contributor

@IvanYashchuk IvanYashchuk commented Dec 13, 2021

In PyTorch/XLA torch.linalg.svd is implemented via _linalg_svd.

The lowering already exists on master; after resolving merge conflicts the only remaining issue is the zero-numel case.

This PR handles empty inputs in _linalg_svd and adds a small C++ test for the output shapes.

Related:

@JackCaoG JackCaoG self-requested a review December 13, 2021 18:32
@IvanYashchuk IvanYashchuk force-pushed the svd branch 3 times, most recently from 62e0bdd to cb354eb Compare December 17, 2021 13:29
@IvanYashchuk IvanYashchuk force-pushed the svd branch 3 times, most recently from ce17370 to d1cc628 Compare January 7, 2022 08:24
@lezcano
Copy link
Copy Markdown
Collaborator

lezcano commented Jan 24, 2022

It would be nice to go over this one again once pytorch/pytorch#69827 is merged.

@IvanYashchuk IvanYashchuk marked this pull request as ready for review May 12, 2022 09:17
@IvanYashchuk IvanYashchuk marked this pull request as draft May 12, 2022 09:17
In PyTorch core the plan is to remove `torch.svd`, it is replaced with
`torch.linalg.svd`. In ATen there are two different operations:
1. `aten::svd` (old)
2. `aten::linalg_svd` (new)

This PR adds XLA lowering for `linalg_svd`.

Resolves pytorch#2755
Unblocks pytorch/pytorch#57772
IvanYashchuk and others added 2 commits May 12, 2022 13:39
Resolve conflicts by keeping current XLA SVD lowering and carrying forward empty-input handling for _linalg_svd.
@IvanYashchuk IvanYashchuk marked this pull request as ready for review May 19, 2026 08:53
Copilot AI review requested due to automatic review settings May 19, 2026 08:53
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a special-case path for torch.linalg.svd lowering on empty XLA tensors, aligning XLA behavior with the newer ATen linalg_svd operator.

Changes:

  • Handles self.numel() == 0 inside _linalg_svd.
  • Constructs empty-result shapes for U, S, and Vh based on full_matrices and compute_uv.
Comments suppressed due to low confidence (3)

torch_xla/csrc/aten_xla_type.cpp:4185

  • The singular values tensor is created with self.options(), so complex inputs produce a complex S. torch.linalg.svd returns real singular values for complex inputs, so this empty-input path should use the corresponding real dtype for s while keeping u/vh in the input dtype.
    auto s = at::zeros(singular_values_sizes, self.options());

torch_xla/csrc/aten_xla_type.cpp:4195

  • When full_matrices is true and n == 0, this returns a non-empty zero matrix for U with shape (..., m, m). The SVD contract expects the returned singular vectors to be orthonormal/unitary; a zero square matrix is not, and it will differ from backends that return an identity basis for this empty-dimension case. Populate the full square factor with an identity basis instead of zeros.
    auto u = at::zeros(u_sizes, self.options());

torch_xla/csrc/aten_xla_type.cpp:4200

  • When full_matrices is true and m == 0, this returns a non-empty zero matrix for Vh with shape (..., n, n). The full singular-vector factor should be unitary/orthonormal for torch.linalg.svd; use an identity basis for the square factor instead of zeros in this empty-dimension case.
    auto vh = at::zeros(vh_sizes, self.options());

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread torch_xla/csrc/aten_xla_type.cpp Outdated
@IvanYashchuk IvanYashchuk changed the title Add lowering for torch.linalg.svd Handle empty inputs in XLA _linalg_svd May 19, 2026
@IvanYashchuk
Copy link
Copy Markdown
Contributor Author

After resolving conflicts against current master, most of the original PR is already upstream: _linalg_svd lowering, codegen registration in codegen/xla_native_functions.yaml, C++ coverage in the split test files, and the OpInfo skip are all present on master already. The only remaining branch delta is the empty-input handling for _linalg_svd, so I renamed the PR to reflect that narrower scope.

@IvanYashchuk
Copy link
Copy Markdown
Contributor Author

Added focused C++ coverage in ab18ba0 for empty _linalg_svd inputs across full_matrices and compute_uv, comparing XLA output shapes against CPU. I did not enable the linalg.svd/linalg.svdvals OpInfo entries here because they are currently disabled on XLA for broader dynamic-slice failures, so that would expand this PR beyond the empty-input fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants