test: strengthen assertions to kill surviving mutants - #46
Merged
Conversation
- Add dataframe= kwarg assertions to create_dataset and
add_examples_to_dataset calls across skip, overwrite, and append modes
- Restore TestUploadOverwrite and TestUploadAppend classes with full
arg-checking including delete URL and headers={'accept': 'application/json'}
- Assert get_dataset is called with the correct dataset name in runner
- Add test for files in experiments_dir to exercise the non-directory
skip path (catches continue->break mutation)
- Add edge-case evaluator tests: json_match invalid/non-object expected,
numeric_tolerance default boundary, top_k K=20 window, membership None,
llm_judge missing result key and schema arg
- Add observability tests: instrument not called on None/empty, default
auto_instrument=True
- Add helpers tests: scalar wrapping and missing-key default
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dataframe=,input_keys=,output_keys=, delete URL, andheaders=kwarg assertions across all threeon_existmodes in uploader tests; restore missingTestUploadOverwriteandTestUploadAppendclassesget_datasetis called with the correct experiment name (notNone) in runner tests; add test exercising the non-directory skip path in_discoverto catchcontinue→breakmutationsjson_matchinvalid/non-object expected,numeric_tolerancedefault boundary values,top_kK=20 window,membershipNoneoutput,llm_judgemissing result key and invalid schema argNone/empty instrumentors and defaultauto_instrument=True