Skip to content

refactor(tests): cmd tests to follow test conventions#3246

Open
agsaru wants to merge 4 commits into
Netflix:masterfrom
agsaru:refactor-pytest-cmd
Open

refactor(tests): cmd tests to follow test conventions#3246
agsaru wants to merge 4 commits into
Netflix:masterfrom
agsaru:refactor-pytest-cmd

Conversation

@agsaru

@agsaru agsaru commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

PR Type

  • Bug fix
  • New feature
  • Core Runtime change (higher bar -- see CONTRIBUTING.md)
  • Docs / tooling
  • Refactoring

Summary

Copilot AI review requested due to automatic review settings June 8, 2026 19:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Refactors existing pytest suites under test/cmd to use function-based tests, tmp_path, and pytest-mock fixtures, improving consistency and reducing reliance on tempfile and unittest.mock.

Changes:

  • Rewrote metaflow diff command tests to use mocker and tmp_path fixtures.
  • Refactored StubGenerator tests into fixture-driven, function-based tests and expanded parametrization.
  • Updated mocking approach from unittest.mock decorators/context managers to pytest-mock.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
test/cmd/diff/test_metaflow_diff.py Refactors diff-related tests to mocker/tmp_path and updates expectations for extract_code_package/perform_diff.
test/cmd/develop/test_stub_generator.py Converts class-based tests to fixtures + standalone tests and updates mocking to pytest-mock.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/cmd/diff/test_metaflow_diff.py Outdated
Comment thread test/cmd/develop/test_stub_generator.py
@greptile-apps

greptile-apps Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR refactors two test files from unittest class-based style to standalone pytest functions, replacing setup_method/teardown_method with fixtures, tempfile.mkdtemp() with tmp_path, and unittest.mock with pytest-mock's mocker. All existing test cases are preserved, renamed with more descriptive identifiers, and some are consolidated under @pytest.mark.parametrize.

  • test_stub_generator.py: The TestStubGenerator class is replaced by a stub_generator fixture and a reusable mock_getmodule factory fixture; the integration test's object comparison is improved from == to is for MetaflowDataFrame/FunctionParameters identity checks.
  • test_metaflow_diff.py: All five class methods are converted to module-level functions; the test_extract_code_package test is updated to mock code.extract() instead of the old code.tarball interface, aligning it with the current implementation.

Confidence Score: 5/5

Pure test refactoring with no changes to production code; all assertions are equivalent or stricter than before.

Every changed line is in test files. The refactoring faithfully preserves all prior assertions, the updated mock interface for code.extract() now matches the actual implementation, and pytest fixture scoping ensures correct teardown. No new test logic is omitted or regressed.

No files require special attention.

Important Files Changed

Filename Overview
test/cmd/develop/test_stub_generator.py Refactored from unittest-class style to standalone pytest functions with fixtures; uses mocker, tmp_path, and parametrize appropriately; all prior test cases preserved with improved naming
test/cmd/diff/test_metaflow_diff.py Refactored to standalone pytest functions using mocker/tmp_path; test_extract_code_package now correctly mocks code.extract() matching the real implementation; op_diff and op_pull remain unused imports

Reviews (2): Last reviewed commit: "replaced '==' with is" | Re-trigger Greptile

Comment thread test/cmd/diff/test_metaflow_diff.py Outdated
@agsaru agsaru marked this pull request as draft June 8, 2026 19:46
@codecov

codecov Bot commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (master@732ca76). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #3246   +/-   ##
=========================================
  Coverage          ?   28.85%           
=========================================
  Files             ?      381           
  Lines             ?    52467           
  Branches          ?     9260           
=========================================
  Hits              ?    15138           
  Misses            ?    36315           
  Partials          ?     1014           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@agsaru agsaru marked this pull request as ready for review June 9, 2026 08:01
@agsaru agsaru requested a review from Copilot June 9, 2026 08:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comment on lines +157 to +169
def test_op_patch_writes_diff_output_to_file(mocker, tmp_path):
"""Test that op_patch successfully writes the generated diff to the specified patch file."""
mock_perform_diff = mocker.patch("metaflow.cmd.code.perform_diff")
mock_perform_diff.return_value = ["diff --git a/file.txt b/file.txt\n"]

patch_file = tmp_path / "patch.patch"

# Act
op_patch(str(tmp_path), str(patch_file))

# Assert
mock_perform_diff.assert_called_once_with(str(tmp_path), output=True)
assert patch_file.read_text() == "diff --git a/file.txt b/file.txt\n"
Comment thread test/cmd/diff/test_metaflow_diff.py
Comment on lines +42 to +51
@pytest.fixture
def mock_getmodule(mocker):
"""Helper fixture to easily mock inspect.getmodule to return a specific module name."""

with patch("inspect.getmodule", return_value=mock_module):
stub = self.generator._generate_function_stub("test_func", test_func)
def _mocker(module_name="test.module"):
mock_module = mocker.Mock()
mock_module.__name__ = module_name
return mocker.patch("inspect.getmodule", return_value=mock_module)

# Should not contain class objects
assert "<class '" not in stub
assert "def test_func(" in stub
assert "test.module.TestClass" in stub
return _mocker
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants