-
-
Notifications
You must be signed in to change notification settings - Fork 290
Expand file tree
/
Copy pathrecord-and-play-testing-guide.mdc
More file actions
125 lines (99 loc) · 8.85 KB
/
Copy pathrecord-and-play-testing-guide.mdc
File metadata and controls
125 lines (99 loc) · 8.85 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
---
description: Comprehensive guide for creating tests using the Record and Play (RnP) framework
globs: ["test-record/**/*.record.test.ts", "test-play/**/*.play.test.ts", "test/**/*.test-harness.ts", "test/**/*.api-test-cases.ts"]
alwaysApply: false
---
This guide provides the complete instructions for creating tests using the project's "Record and Play" (RnP) framework. It is intended to be used by LLM agents to autonomously create, run, and maintain tests for new and existing features. Adherence to these rules is not optional; it is critical for the stability and predictability of the testing suite.
This framework applies to any major feature area, including `chains` (e.g., `solana`) and `connectors` (e.g., `raydium`). The `rnpExample` directory is the canonical source for all examples referenced below.
## 1. Philosophy: Why Record and Play?
The RnP framework is designed to create high-fidelity tests that are both robust and easy to maintain. The core philosophy is:
* **Record against reality:** Tests are initially run against live, real-world services to record their actual responses. This captures the true behavior of our dependencies.
* **"Play" in isolation:** Once recorded, tests are run in a "Play" mode that uses the saved recordings (mocks) instead of making live network calls. This makes the tests fast, deterministic, and isolated from external failures.
* **Confidence:** This approach ensures that our application logic is tested against realistic data, which gives us high confidence that it will work correctly in production. It prevents "mock drift," where mocks no longer reflect the reality of the API they are simulating.
## 2. Core Components
The RnP framework consists of four key types of files that work together for a given feature.
### `*.test-harness.ts` - The Engine
* **Location:** `test/{chains|connectors}/{feature-name}/{feature-name}.test-harness.ts`
* **Example:** `test/rnpExample/rnpExample.test-harness.ts`
* **Purpose:** The harness is the central engine of a test suite. It is responsible for initializing the application instance under test and, most importantly, defining the `dependencyContracts`.
* **The `dependencyContracts` Object:** This object is a manifest of all external dependencies. As seen in `rnpExample.test-harness.ts`, it maps a human-readable name (e.g., `dep1_A`) to a dependency contract. This mapping is what allows the framework to intercept calls for recording or mocking.
### `*.api-test-cases.ts` - The Single Source of Truth
* **Location:** `test/{chains|connectors}/{feature-name}/{feature-name}.api-test-cases.ts`
* **Example:** `test/rnpExample/rnpExample.api-test-cases.ts`
* **Purpose:** This file defines the specific API requests that will be used for both recording and testing. By defining test cases in one place (e.g., `export const callABC = new TestCase(...)`), we prevent code duplication and ensure the "Record" and "Play" tests are perfectly aligned.
### `*.record.test.ts` - The "Record" Tests
* **Location:** `test-record/{chains|connectors}/{feature-name}/`
* **Example:** `test-record/rnpExample/rnpExample.record.test.ts`
* **Purpose:** The sole responsibility of this test suite is to generate mock files. It imports a `TestCase` and uses the `createRecordTest(harness)` method to generate one-line `it` blocks that executes the test against live services.
* **Execution:** "Record" tests are slow, make real network calls, and should be run individually using the `pnpm test-record -t "your exact test name"` command.
### `*.play.test.ts` - The "Play" Tests
* **Location:** `test-play/{chains|connectors}/{feature-name}/`
* **Example:** `test-play/rnpExample/rnpExample.play.test.ts`
* **Purpose:** This is the fast, isolated unit test suite. It imports a `TestCase` and uses the `createPlayTest(harness)` method to generate one-line `it` blocks that validate application logic against generated mocks.
* **Execution:** These tests are fast and run as part of the main `pnpm test` suite.
## 3. Directory and Naming Conventions
The framework relies on a strict set of conventions. These are functional requirements, not style suggestions. The `rnpExample` has a simplified API source structure and should not be followed for new development; use the `solana` or `raydium` structure as your template.
### Directory Structure for Chains
* **Source Code:** `src/chains/solana/`
* **Shared Test Artifacts:** `test/chains/solana/` (for harness and API test cases)
* **"Play" Tests:** `test-play/chains/solana/`
* **"Record" Tests:** `test-record/chains/solana/`
### Directory Structure for Connectors
* **Source Code:** `src/connectors/raydium/`
* **Shared Test Artifacts:** `test/connectors/raydium/` (for harness and API test cases)
* **"Play" Tests:** `test-play/connectors/raydium/`
* **"Record" Tests:** `test-record/connectors/raydium/`
### Test Naming
The `describe()` and `it()` block names in the "Record" and "Play" tests **MUST MATCH EXACTLY**. The test case's variable name is used for the `it` block's name to ensure consistency. Compare the `it('callABC', ...)` blocks in `rnpExample.record.test.ts` and `rnpExample.play.test.ts` to see this in practice. This naming convention is how Jest associates API response snapshots with the correct test.
### Command Segregation
* **To run all fast "Play" tests:**
```bash
pnpm test-play
```
* **To run a slow "Record" test and generate mocks:**
```bash
pnpm test-record -t "your exact test name"
```
## 4. The Critical Rule of Dependency Management
To ensure unit tests are fast and never make accidental live network calls, the framework enforces a strict safety policy. Understanding this is not optional.
* When you add even one method from a dependency object (e.g., `Dependency1.A_basicMethod`) to the `dependencyContracts` in `rnpExample.test-harness.ts`, the *entire object* (`dep1`) is now considered "managed."
* **In "Record" Mode:** Managed dependencies behave as expected. The listed methods are spied on, and unlisted methods on the same object (like `Dependency1.unmappedMethod`) call their real implementation.
* **In "Play" Mode:** This is where the safety rule applies. **Every method** on a managed object must be explicitly mocked. If a method (like `Dependency1.unmappedMethod`) is called without a mock, the test will fail. The `callUnmappedMethodMocked` test case in `rnpExample.api-test-cases.ts` is designed specifically to validate this failure.
* **Why?** This strictness forces the agent developer to be fully aware of every interaction with an external service. It makes it impossible for a dependency to add a new network call that would slow down unit tests.
* **Truly Unmanaged Dependencies:** In contrast, `dep2` from `rnpExample` is not mentioned in the `dependencyContracts`. It is "unmanaged," so its methods can be called freely in either mode. The `callUnlistedDep` test case demonstrates this.
## 5. Workflow: Adding a New Endpoint Test
Follow this step-by-step process. In these paths, `{feature-type}` is either `chains` or `connectors`, and `{feature-name}` is the name of your chain or connector (e.g., `solana`, `raydium`).
**Step 1: Define the Test Case**
* Open or create `test/{feature-type}/{feature-name}/{feature-name}.api-test-cases.ts`.
* Create and export a new `TestCase` instance (e.g., `export const callNewFeature = new TestCase(...)`).
**Step 2: Update the Test Harness**
* Open or create `test/{feature-type}/{feature-name}/{feature-name}.test-harness.ts`.
* If your endpoint uses any new dependencies, add them to `dependencyContracts`, as seen in `rnpExample.test-harness.ts`.
**Step 3: Create the "Record" Test**
* Open or create `test-record/{feature-type}/{feature-name}/{feature-name}.record.test.ts`.
* Add a new one-line `it()` block using the `createRecordTest` helper:
```typescript
it('your_test_case_name', yourTestCase.createRecordTest(harness));
```
**Step 4: Run the "Record" Test**
* Execute the "Record" test from your terminal to generate mock and snapshot files.
```bash
pnpm test-record {feature-name}.record.test.ts -t "your test case name"
```
**Step 5: Create the "Play" Test**
* Open or create `test-play/{feature-type}/{feature-name}/{feature-name}.play.test.ts`.
* Add a new one-line `it()` block that **exactly matches** the "Record" test:
```typescript
it('your_test_case_name', yourTestCase.createPlayTest(harness));
```
**Step 6: Run the "Play" Test**
* Execute the main test suite to verify your logic against the generated mocks.
```bash
pnpm test-play {feature-name}.play.test.ts
```
* The test will run, using the mocks you generated. It will pass if the application logic correctly processes the mocked dependency responses to produce the expected final API response.
**Step 7: Run the whole unit test suite**
* Execute the mocked test suite to verify no breaking changes were introduced.
```bash
pnpm test
```