Skip to content

default_transforms() raises unhandled ZeroDivisionError on an empty documents listΒ #2871

Description

@mittalpk

Describe the bug

ragas.testset.transforms.default.default_transforms() computes a document-length histogram and divides each bin count by len(documents) before validating that documents is non-empty:

bin_ranges = [(0, 100), (101, 500), (501, float("inf"))]
result = count_doc_length_bins(documents, bin_ranges)
result = {k: v / len(documents) for k, v in result.items()}

If documents is an empty list, this raises a raw ZeroDivisionError: division by zero instead of a clear, actionable message.

This is reachable from the public API: TestsetGenerator.generate_with_langchain_docs() and generate_with_llamaindex_docs() both call default_transforms(documents=..., ...) whenever the caller doesn't pass an explicit transforms argument, so passing an empty documents list (or a list that resolves to empty after upstream filtering) crashes with this opaque error.

Note the function already validates one bad-input case explicitly β€” "too short" documents raise a clear ValueError:

else:
    raise ValueError(
        "Documents appears to be too short (ie 100 tokens or less). Please provide longer documents."
    )

The empty-list case just isn't covered by that same validation, since the crash happens earlier, during the bin-count division.

Ragas version

Reproduced against current main (298b682).

Reproduction

from ragas.testset.transforms.default import default_transforms

default_transforms(documents=[], llm=my_llm, embedding_model=my_embeddings)
# ZeroDivisionError: division by zero

Expected behavior

A clear ValueError (matching the existing "too short" validation pattern), e.g. "The documents list is empty. Please provide at least one document to generate default transforms for."

I have a fix + regression tests ready and will open a PR shortly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions