Skip to content

Commit ba128f0

Browse files
authored
Fixes for IDP tools (#159)
* Fixes for IDP tools * Fixes tests * Fixes docs * Fix README
1 parent 5d37bb3 commit ba128f0

4 files changed

Lines changed: 411 additions & 94 deletions

File tree

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
## Harness MCP Server 2.0
22

3-
An MCP (Model Context Protocol) server that gives AI agents full access to the Harness.io platform through 11 consolidated tools and 168 resource types.
3+
An MCP (Model Context Protocol) server that gives AI agents full access to the Harness.io platform through 11 consolidated tools and 169 resource types.
44

55
## Why Use This MCP Server
66

77
Most MCP servers map one tool per API endpoint. For a platform as broad as Harness, that means 240+ tools — and LLMs get worse at tool selection as the count grows. Context windows fill up with schemas, and every new endpoint means new code.
88

99
This server is built differently:
1010

11-
- **11 tools, 168 resource types.** A registry-based dispatch system routes `harness_list`, `harness_get`, `harness_create`, etc. to any Harness resource — pipelines, services, environments, orgs, projects, feature flags, cost data, and more. The LLM picks from 11 tools instead of hundreds.
11+
- **11 tools, 169 resource types.** A registry-based dispatch system routes `harness_list`, `harness_get`, `harness_create`, etc. to any Harness resource — pipelines, services, environments, orgs, projects, feature flags, cost data, and more. The LLM picks from 11 tools instead of hundreds.
1212
- **Full platform coverage.** 31 toolsets spanning CI/CD, GitOps, Feature Flags, Cloud Cost Management, Security Testing, Chaos Engineering, Database DevOps, Internal Developer Portal, Software Supply Chain, Governance, Service Overrides, Visualizations, and more. Not just pipelines — the entire Harness platform.
1313
- **Multi-project workflows out of the box.** Agents discover organizations and projects dynamically — no hardcoded env vars needed. Ask "show failed executions across all projects" and the agent can navigate the full account hierarchy.
1414
- **30 prompt templates.** Pre-built prompts for common workflows: build & deploy apps end-to-end, debug failed pipelines, review DORA metrics, triage vulnerabilities, optimize cloud costs, audit access control, plan feature flag rollouts, review pull requests, approve pending pipelines, and more.
@@ -954,7 +954,7 @@ Harness pipelines can be stored in three ways:
954954

955955
## Resource Types
956956

957-
168 resource types organized across 31 toolsets. Each resource type supports a subset of CRUD operations and optional execute actions.
957+
169 resource types organized across 31 toolsets. Each resource type supports a subset of CRUD operations and optional execute actions.
958958

959959
### Platform
960960

@@ -1491,7 +1491,7 @@ Available toolset names:
14911491
+--------v---------+
14921492
| Registry | <-- Declarative resource definitions
14931493
| 32 Toolsets | (data files, not code)
1494-
| 168 Resource Types|
1494+
| 169 Resource Types|
14951495
+--------+---------+
14961496
|
14971497
+--------v---------+
Lines changed: 17 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,34 +1,33 @@
1-
# Test Plan: IDP Score (`idp_score`)
1+
# Test Plan: IDP Score (`idp_score`, `idp_score_summary`)
22

33
| Field | Value |
44
|-------|-------|
5-
| **Resource Type** | `idp_score` |
6-
| **Display Name** | IDP Score |
5+
| **Resource Type** | `idp_score`, `idp_score_summary` |
6+
| **Display Name** | IDP Score, IDP Score Summary |
77
| **Toolset** | idp |
88
| **Scope** | account |
9-
| **Operations** | list, get |
9+
| **Operations** | `idp_score`: list; `idp_score_summary`: get |
1010
| **Execute Actions** | None |
11-
| **Identifier Fields** | entity_id |
12-
| **Filter Fields** | None |
11+
| **Identifier Fields** | entity_identifier |
12+
| **Filter Fields** | entity_identifier |
1313
| **Deep Link** | No |
1414

1515
## Test Cases
1616

1717
| Test ID | Category | Description | Prompt | Expected Result |
1818
|---------|----------|-------------|--------|-----------------|
19-
| TC-idp_score-001 | List | List all entity scores with defaults | `harness_list(resource_type="idp_score")` | Returns paginated list of entity scores |
20-
| TC-idp_score-002 | List | List with pagination page 0 | `harness_list(resource_type="idp_score", page=0, size=5)` | Returns first page with up to 5 scores |
21-
| TC-idp_score-003 | List | List with pagination page 1 | `harness_list(resource_type="idp_score", page=1, size=5)` | Returns second page of scores |
22-
| TC-idp_score-004 | List | List with large page size | `harness_list(resource_type="idp_score", size=100)` | Returns up to 100 scores |
23-
| TC-idp_score-005 | Get | Get score by entity_id | `harness_get(resource_type="idp_score", entity_id="my-service")` | Returns score summary for the entity |
24-
| TC-idp_score-006 | Get | Verify score response structure | `harness_get(resource_type="idp_score", entity_id="my-service")` | Response contains score summary fields |
25-
| TC-idp_score-007 | Error | Get with missing entity_id | `harness_get(resource_type="idp_score")` | Error: entity_id is required |
26-
| TC-idp_score-008 | Error | Get non-existent entity score | `harness_get(resource_type="idp_score", entity_id="nonexistent")` | Error: entity not found (404) |
27-
| TC-idp_score-009 | Edge | List with page beyond data | `harness_list(resource_type="idp_score", page=9999)` | Returns empty list |
28-
| TC-idp_score-010 | Edge | List with size=1 | `harness_list(resource_type="idp_score", size=1)` | Returns exactly 1 score |
19+
| TC-idp_score-001 | List | List scorecard scores for an entity | `harness_list(resource_type="idp_score", filters={"entity_identifier":"default/Component/my-service"})` | Returns overall_score plus scorecard score items for the entity |
20+
| TC-idp_score-002 | List | List scorecard scores using top-level entity_identifier | `harness_list(resource_type="idp_score", entity_identifier="default/Component/my-service")` | Returns overall_score plus scorecard score items for the entity |
21+
| TC-idp_score-003 | Error | List without entity_identifier | `harness_list(resource_type="idp_score")` | Error from IDP API or validation indicating entity_identifier is required |
22+
| TC-idp_score-004 | Error | List scorecard scores for non-existent entity | `harness_list(resource_type="idp_score", filters={"entity_identifier":"default/Component/nonexistent"})` | Error: entity not found or empty/no scores depending on API behavior |
23+
| TC-idp_score_summary-001 | Get | Get aggregate score summary for an entity | `harness_get(resource_type="idp_score_summary", resource_id="default/Component/my-service")` | Returns aggregate score summary for the entity |
24+
| TC-idp_score_summary-002 | Get | Get aggregate score summary via params | `harness_get(resource_type="idp_score_summary", params={"entity_identifier":"default/Component/my-service"})` | Returns aggregate score summary for the entity |
25+
| TC-idp_score_summary-003 | Error | Get summary without entity_identifier | `harness_get(resource_type="idp_score_summary")` | Error: entity_identifier is required |
26+
| TC-idp_score_summary-004 | Error | Get summary for non-existent entity | `harness_get(resource_type="idp_score_summary", resource_id="default/Component/nonexistent")` | Error: entity not found or empty/no summary depending on API behavior |
2927

3028
## Notes
3129
- Account-scoped resource
32-
- API paths: GET `/v1/scores` (list), GET `/v1/scores/{entityId}` (get)
33-
- No filter fields; only pagination params supported for list
30+
- API paths: GET `/v1/scores` (`idp_score` list), GET `/v1/scores/summary` (`idp_score_summary` get)
31+
- `idp_score` requires `entity_identifier` and does not expose pagination
32+
- `idp_score_summary` requires `entity_identifier`
3433
- No deep link template defined

docs/testing/idp_score/test_report.md

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
# Test Report: IDP Score (`idp_score`)
1+
# Test Report: IDP Score (`idp_score`, `idp_score_summary`)
22

33
| Field | Value |
44
|-------|-------|
5-
| **Resource Type** | `idp_score` |
5+
| **Resource Type** | `idp_score`, `idp_score_summary` |
66
| **Date** | 2026-03-23 |
77
| **Tester** | MCP Automated Test |
88
| **Account ID** | px7xd_BFRCi-pfWPYXVjvw |
@@ -13,26 +13,24 @@
1313

1414
| Test ID | Description | Prompt | Expected Result | Status | Actual Result | Notes |
1515
|---------|-------------|--------|-----------------|--------|---------------|-------|
16-
| TC-idp_score-001 | List all entity scores with defaults | `harness_list(resource_type="idp_score")` | Returns paginated list of entity scores | ✅ Passed | Returns entity scores (requires entity_identifier filter) | Requires entity_identifier filter |
17-
| TC-idp_score-002 | List with pagination page 0 | `harness_list(resource_type="idp_score", page=0, size=5)` | Returns first page with up to 5 scores | ⬜ Pending | | |
18-
| TC-idp_score-003 | List with pagination page 1 | `harness_list(resource_type="idp_score", page=1, size=5)` | Returns second page of scores | ⬜ Pending | | |
19-
| TC-idp_score-004 | List with large page size | `harness_list(resource_type="idp_score", size=100)` | Returns up to 100 scores | ⬜ Pending | | |
20-
| TC-idp_score-005 | Get score by entity_id | `harness_get(resource_type="idp_score", entity_id="my-service")` | Returns score summary for the entity | ⬜ Pending | | |
21-
| TC-idp_score-006 | Verify score response structure | `harness_get(resource_type="idp_score", entity_id="my-service")` | Response contains score summary fields | ⬜ Pending | | |
22-
| TC-idp_score-007 | Get with missing entity_id | `harness_get(resource_type="idp_score")` | Error: entity_id is required | ⬜ Pending | | |
23-
| TC-idp_score-008 | Get non-existent entity score | `harness_get(resource_type="idp_score", entity_id="nonexistent")` | Error: entity not found (404) | ⬜ Pending | | |
24-
| TC-idp_score-009 | List with page beyond data | `harness_list(resource_type="idp_score", page=9999)` | Returns empty list | ⬜ Pending | | |
25-
| TC-idp_score-010 | List with size=1 | `harness_list(resource_type="idp_score", size=1)` | Returns exactly 1 score | ⬜ Pending | | |
16+
| TC-idp_score-001 | List scorecard scores for an entity | `harness_list(resource_type="idp_score", filters={"entity_identifier":"default/Component/my-service"})` | Returns overall_score plus scorecard score items for the entity | ⬜ Pending | | |
17+
| TC-idp_score-002 | List scorecard scores using top-level entity_identifier | `harness_list(resource_type="idp_score", entity_identifier="default/Component/my-service")` | Returns overall_score plus scorecard score items for the entity | ⬜ Pending | | |
18+
| TC-idp_score-003 | List without entity_identifier | `harness_list(resource_type="idp_score")` | Error from IDP API or validation indicating entity_identifier is required | ⬜ Pending | | |
19+
| TC-idp_score-004 | List scorecard scores for non-existent entity | `harness_list(resource_type="idp_score", filters={"entity_identifier":"default/Component/nonexistent"})` | Error: entity not found or empty/no scores depending on API behavior | ⬜ Pending | | |
20+
| TC-idp_score_summary-001 | Get aggregate score summary for an entity | `harness_get(resource_type="idp_score_summary", resource_id="default/Component/my-service")` | Returns aggregate score summary for the entity | ⬜ Pending | | |
21+
| TC-idp_score_summary-002 | Get aggregate score summary via params | `harness_get(resource_type="idp_score_summary", params={"entity_identifier":"default/Component/my-service"})` | Returns aggregate score summary for the entity | ⬜ Pending | | |
22+
| TC-idp_score_summary-003 | Get summary without entity_identifier | `harness_get(resource_type="idp_score_summary")` | Error: entity_identifier is required | ⬜ Pending | | |
23+
| TC-idp_score_summary-004 | Get summary for non-existent entity | `harness_get(resource_type="idp_score_summary", resource_id="default/Component/nonexistent")` | Error: entity not found or empty/no summary depending on API behavior | ⬜ Pending | | |
2624

2725
## Summary
2826

2927
| Metric | Count |
3028
|--------|-------|
31-
| Total Tests | 10 |
32-
| ✅ Passed | 1 |
29+
| Total Tests | 8 |
30+
| ✅ Passed | 0 |
3331
| ❌ Failed | 0 |
3432
| ⚠️ Blocked | 0 |
35-
| ⬜ Not Run | 9 |
33+
| ⬜ Not Run | 8 |
3634

3735
## Issues Found
3836

0 commit comments

Comments
 (0)