Skip to content

Split utils key and JSON helpers#1686

Open
vmoens wants to merge 1 commit intopytorch:mainfrom
vmoens:split-utils-key-json-helpers
Open

Split utils key and JSON helpers#1686
vmoens wants to merge 1 commit intopytorch:mainfrom
vmoens:split-utils-key-json-helpers

Conversation

@vmoens
Copy link
Copy Markdown
Collaborator

@vmoens vmoens commented Apr 29, 2026

Summary

  • move filesystem key encoding and robust-key helpers into tensordict._utils_key_json
  • move JSON backend helpers into the same internal module
  • keep tensordict.utils re-exporting the old names with __module__ == "tensordict.utils"
  • add focused tests for import identity, key round trip/warnings, and JSON serialization

BC notes

  • imports from tensordict.utils remain valid
  • helper function module paths remain tensordict.utils
  • JSON backend registration still occurs when tensordict.utils is imported

Test plan

  • python -m pytest test/test_utils_key_json_split.py -q
  • ulimit -n 4096 && python -m pytest test/test_tensordict.py -k "robust_key or memmap" -q
  • python -m compileall -q tensordict/utils.py tensordict/_utils_key_json.py
  • import smoke test for _encode_key_for_filesystem and json_dumps module paths

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 29, 2026
@github-actions github-actions Bot added the Test label Apr 29, 2026
@github-actions
Copy link
Copy Markdown
Contributor

PR Title Label Error

PR title must start with a label prefix in brackets (e.g., [BugFix]).

Current title: Split utils key and JSON helpers

Supported Prefixes

Your PR title must start with exactly one of these prefixes (case-insensitive):

Prefix Label Applied Example
[BugFix] or [Fix] bug [BugFix] Fix memory leak in TensorDict
[Feature] Feature [Feature] Add new storage backend
[Doc] or [Docs] documentation [Doc] Update installation guide
[Refactor] Refactor [Refactor] Clean up module imports
[CI] CI [CI] Fix workflow permissions
[Test] or [Tests] Test [Test] Add unit tests for nn module
[Compile] Compile [Compile] Fix torch.compile issue
[Performance] or [Perf] Performance [Perf] Optimize tensor operations
[Deprecation] Deprecation [Deprecation] Mark old function
[Setup] setup [Setup] Update build configuration
[Distributed] or [Dist] Distributed [Distributed] Add scatter collective
[Benchmark] or [Bench] Benchmarks [Benchmark] Add compile benchmark
[Typing] or [Type] Typing [Typing] Add type stubs
[BC-breaking] or [BC] BC-breaking [BC-breaking] Remove deprecated API
[Formatting] or [Format] Formatting [Format] Fix code style
[Quality] Quality [Quality] Improve error messages

Note: Matching is case-insensitive. Common variations (singular/plural) are supported.

@vmoens vmoens force-pushed the split-utils-key-json-helpers branch from 62decdb to 706f5d3 Compare April 29, 2026 20:07
@github-actions
Copy link
Copy Markdown
Contributor

PR Title Label Error

PR title must start with a label prefix in brackets (e.g., [BugFix]).

Current title: Split utils key and JSON helpers

Supported Prefixes

Your PR title must start with exactly one of these prefixes (case-insensitive):

Prefix Label Applied Example
[BugFix] or [Fix] bug [BugFix] Fix memory leak in TensorDict
[Feature] Feature [Feature] Add new storage backend
[Doc] or [Docs] documentation [Doc] Update installation guide
[Refactor] Refactor [Refactor] Clean up module imports
[CI] CI [CI] Fix workflow permissions
[Test] or [Tests] Test [Test] Add unit tests for nn module
[Compile] Compile [Compile] Fix torch.compile issue
[Performance] or [Perf] Performance [Perf] Optimize tensor operations
[Deprecation] Deprecation [Deprecation] Mark old function
[Setup] setup [Setup] Update build configuration
[Distributed] or [Dist] Distributed [Distributed] Add scatter collective
[Benchmark] or [Bench] Benchmarks [Benchmark] Add compile benchmark
[Typing] or [Type] Typing [Typing] Add type stubs
[BC-breaking] or [BC] BC-breaking [BC-breaking] Remove deprecated API
[Formatting] or [Format] Formatting [Format] Fix code style
[Quality] Quality [Quality] Improve error messages

Note: Matching is case-insensitive. Common variations (singular/plural) are supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant