[CI Fix] Propagate dtype in TorchAOBaseTensor._to_copy#4358
Closed
jainapurva wants to merge 3 commits intomainfrom
Closed
[CI Fix] Propagate dtype in TorchAOBaseTensor._to_copy#4358jainapurva wants to merge 3 commits intomainfrom
jainapurva wants to merge 3 commits intomainfrom
Conversation
## Problem PR #4297 added `non_blocking` propagation to TorchAOBaseTensor._to_copy, but introduced a bug: while `_get_to_kwargs` returns `device`, `dtype`, and `non_blocking`, the `_to_copy` handler only propagated `device` and `non_blocking` to inner tensors. This meant that calls like `tensor.to(dtype=torch.float16)` or `tensor.to(device='cuda', dtype=torch.bfloat16)` would change the wrapper tensor's dtype but NOT the inner tensors (qdata, scale, etc.), causing a dtype mismatch between the wrapper and its data. ## Fix - Pop `dtype` from kwargs and pass it to all inner `.to()` calls - Use explicit keyword arguments for clarity: `device=device, dtype=dtype, non_blocking=non_blocking` This ensures all three parameters are consistently propagated to inner tensors when calling `.to()` on TorchAOBaseTensor subclasses. ## Testing Added `test_to_copy_propagates_dtype_and_non_blocking` to verify: - Dtype-only changes propagate correctly - Combined device + dtype + non_blocking changes work - All existing tests continue to pass Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/4358
Note: Links to docs will display an error until the docs builds have been completed. ❌ 6 New FailuresAs of commit 23df18a with merge base 28e6aca ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
jainapurva
added
the
module: not user facing
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…hange ## Problem PyTorch nightly now implements saturated casting to float8_e4m3fn in eager mode, matching the behavior that was previously only in compiled/triton mode. The test `test_cast_to_float8_e4m3fn_saturation_behavior` was expecting the old unsaturated behavior (out-of-range values → NaN), causing H100 CI failures. ## Fix Updated the test to verify the new saturated casting behavior: - Changed assertion from expecting NaN to expecting saturation - Added verification that out-of-range values are clamped to max_val - Updated assertions to verify eager and compiled modes produce identical results - Updated comments to reflect the completed TODO from issue #1912 ## Testing This fixes the H100 test failures on main branch where the test was asserting: ```python assert torch.all(torch.isnan(data_out_of_range_f8)) # Old behavior ``` But PyTorch now produces saturated values (448/-448) instead of NaN. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PR #4297 added
non_blockingpropagation to TorchAOBaseTensor._to_copy, but introduced a bug: while_get_to_kwargsreturnsdevice,dtype, andnon_blocking, the_to_copyhandler only propagateddeviceandnon_blockingto inner tensors.This meant that calls like
tensor.to(dtype=torch.float16)ortensor.to(device='cuda', dtype=torch.bfloat16)would change the wrapper tensor's dtype but NOT the inner tensors (qdata, scale, etc.), causing a dtype mismatch between the wrapper and its data.Fix
dtypefrom kwargs and pass it to all inner.to()callsdevice=device, dtype=dtype, non_blocking=non_blockingThis ensures all three parameters are consistently propagated to inner tensors when calling
.to()on TorchAOBaseTensor subclasses.Testing
Added
test_to_copy_propagates_dtype_and_non_blockingto verify: