Skip to content

[moe training] default pad_token_groups_for_grouped_mm=False#4080

Merged
danielvegamyhre merged 1 commit intomainfrom
defaultoff
Mar 14, 2026
Merged

[moe training] default pad_token_groups_for_grouped_mm=False#4080
danielvegamyhre merged 1 commit intomainfrom
defaultoff

Conversation

@danielvegamyhre
Copy link
Copy Markdown
Contributor

@danielvegamyhre danielvegamyhre commented Mar 13, 2026

Summary

  • Default pad_token_groups_for_grouped_mm to False to avoid surprising users with the extra pad/unpad kernels that incur overhead. This is often handled upstream of the quantization + grouped MM step now, by systems like HybridEP etc, so best for the user to explicitly request the padding via the flag if they need it.
  • Update tests and benchmarks accordingly

Tests

  • pytest test/prototype/moe_training/test_training.py -s

@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Mar 13, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/4080

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit b6bf1a5 with merge base eb64bfb (image):

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 13, 2026
@danielvegamyhre danielvegamyhre force-pushed the defaultoff branch 2 times, most recently from f6a27bd to 10362fc Compare March 13, 2026 20:16
@danielvegamyhre danielvegamyhre added mx module: training quantize_ api training flow labels Mar 13, 2026
@danielvegamyhre danielvegamyhre added this to the MXFP8 Training milestone Mar 13, 2026
@danielvegamyhre danielvegamyhre force-pushed the defaultoff branch 3 times, most recently from 7911823 to 5eb9e5e Compare March 13, 2026 22:29
@danielvegamyhre danielvegamyhre merged commit 960f307 into main Mar 14, 2026
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: training quantize_ api training flow mx

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants