Describe the bug
ragas.testset.transforms.default.default_transforms() computes a document-length histogram and divides each bin count by len(documents) before validating that documents is non-empty:
bin_ranges = [(0, 100), (101, 500), (501, float("inf"))]
result = count_doc_length_bins(documents, bin_ranges)
result = {k: v / len(documents) for k, v in result.items()}
If documents is an empty list, this raises a raw ZeroDivisionError: division by zero instead of a clear, actionable message.
This is reachable from the public API: TestsetGenerator.generate_with_langchain_docs() and generate_with_llamaindex_docs() both call default_transforms(documents=..., ...) whenever the caller doesn't pass an explicit transforms argument, so passing an empty documents list (or a list that resolves to empty after upstream filtering) crashes with this opaque error.
Note the function already validates one bad-input case explicitly β "too short" documents raise a clear ValueError:
else:
raise ValueError(
"Documents appears to be too short (ie 100 tokens or less). Please provide longer documents."
)
The empty-list case just isn't covered by that same validation, since the crash happens earlier, during the bin-count division.
Ragas version
Reproduced against current main (298b682).
Reproduction
from ragas.testset.transforms.default import default_transforms
default_transforms(documents=[], llm=my_llm, embedding_model=my_embeddings)
# ZeroDivisionError: division by zero
Expected behavior
A clear ValueError (matching the existing "too short" validation pattern), e.g. "The documents list is empty. Please provide at least one document to generate default transforms for."
I have a fix + regression tests ready and will open a PR shortly.
Describe the bug
ragas.testset.transforms.default.default_transforms()computes a document-length histogram and divides each bin count bylen(documents)before validating thatdocumentsis non-empty:If
documentsis an empty list, this raises a rawZeroDivisionError: division by zeroinstead of a clear, actionable message.This is reachable from the public API:
TestsetGenerator.generate_with_langchain_docs()andgenerate_with_llamaindex_docs()both calldefault_transforms(documents=..., ...)whenever the caller doesn't pass an explicittransformsargument, so passing an emptydocumentslist (or a list that resolves to empty after upstream filtering) crashes with this opaque error.Note the function already validates one bad-input case explicitly β "too short" documents raise a clear
ValueError:The empty-list case just isn't covered by that same validation, since the crash happens earlier, during the bin-count division.
Ragas version
Reproduced against current
main(298b682).Reproduction
Expected behavior
A clear
ValueError(matching the existing "too short" validation pattern), e.g. "The documents list is empty. Please provide at least one document to generate default transforms for."I have a fix + regression tests ready and will open a PR shortly.