Skip to content

Add full mock-provider E2E regression suite and workflowΒ #13

Description

@Xunzhuo

Goal

Build a complete E2E regression suite with a mock LLM provider server and GitHub workflow coverage for CLI, Gateway, and core user journeys.

Scope

  • Mock LLM provider server with deterministic chat, streaming, tool-call, error, retry, and degraded-response scenarios.
  • Full CLI regression for setup, chat, memory updates/search, tools, provider config, cron, and recovery paths.
  • Gateway regression for supported IM adapters, duplicate delivery, async replies, cron delivery, and failure messaging.
  • Workflow that runs the high-signal E2E suite in CI without requiring live provider credentials.
  • Clear separation between fast PR gate, full E2E gate, and optional live certification.

Acceptance Criteria

  • A reusable mock provider server is available for E2E tests.
  • CLI and Gateway core flows have deterministic E2E coverage.
  • The workflow runs in GitHub Actions without external LLM credentials.
  • Failure output points to the broken surface and scenario.
  • Existing release/live certification tests are not used as a substitute for mock-provider E2E regression.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/cliCommand-line interface behaviorarea/e2eEnd-to-end testing and workflow coveragearea/imInstant messaging and gateway modesarea/qualityCode quality, cleanup, and regression hardeningenhancementNew feature or requestkind/epicRoadmap-sized work that may need decompositionpriority/p0Critical roadmap blocker

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions