Skip to content

Add mixed granularity tests for smooth quant #3847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cyxlily wants to merge 9 commits into
base: main
Choose a base branch
from
+7 −2
Open

Add mixed granularity tests for smooth quant #3847

Changes from 7 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
5183287
Add mixed granularity tests for smooth quant
cyxlily Feb 10, 2026
2fcda9b
Refine codes
cyxlily Feb 10, 2026
0196250
Merge branch 'pytorch:main' into smooth_quant_test
cyxlily Mar 18, 2026
546e05f
Merge branch 'pytorch:main' into smooth_quant_test
cyxlily Mar 20, 2026
55c56d0
Merge remote-tracking branch 'upstream/main' into smooth_quant_test
cyxlily Mar 30, 2026
6a4c73e
Remove redundant test
cyxlily Mar 31, 2026
501e4c5
Merge remote-tracking branch 'upstream/main' into smooth_quant_test
cyxlily Apr 24, 2026
2d237ad
Refine codes
cyxlily Apr 27, 2026
85c52e1
Use list
cyxlily Apr 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion test/prototype/test_smoothquant.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,17 @@ def setUpClass(cls):
"base_config",
[
Int8DynamicActivationInt8WeightConfig(version=2),
Int8StaticActivationInt8WeightConfig(granularity=PerRow()),
Int8DynamicActivationInt8WeightConfig(version=2, granularity=PerTensor()),
Comment thread
jerryzh168 marked this conversation as resolved.
Outdated
Int8DynamicActivationInt8WeightConfig(
version=2, granularity=[PerRow(), PerTensor()]
Copy link
Copy Markdown
Contributor

@jerryzh168 jerryzh168 Apr 24, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually why is this a list?

I saw type is changed to

ao/torchao/quantization/quant_api.py

Line 856 in 9058b58

Union[Granularity, Tuple[Granularity, Granularity], list[Granularity]]

I feel we should not have both tuple and list, can you have a followup PR to change this to Union[Granularity, Tuple[Granularity, Granularity]] to reduce the complexity here?

Copy link
Copy Markdown
Contributor Author

@cyxlily cyxlily Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for comments, updated the UT.
For the followup PR, the JSON encoder explicitly raises NotImplementedError for tuples: https://github.com/pytorch/ao/blob/main/torchao/core/config.py#L146
So even though users can pass a Tuple, internally it still needs to be stored as a list for serialization.
Does it still make sense to change the API from list to tuple?

Copy link
Copy Markdown
Contributor

@jerryzh168 jerryzh168 Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh I see, in that case we should remove tuple and just use List, I've seen the json error before as well

),
Int8DynamicActivationInt8WeightConfig(
version=2, granularity=[PerTensor(), PerRow()]
),
Int8StaticActivationInt8WeightConfig(),
Int8StaticActivationInt8WeightConfig(granularity=PerTensor()),
Int8StaticActivationInt8WeightConfig(granularity=[PerRow(), PerTensor()]),
Int8StaticActivationInt8WeightConfig(granularity=[PerTensor(), PerRow()]),
# Note: float8_static_activation_float8_weight is broken after recent PyTorch update.
# TODO(#1639): Fix for supporting more API in torchao/quantization/quant_api.py
],
Expand Down
Loading