Teradata lineage and dependency graph analysis

[earthshiner]

Summary

This PR introduces a complete graph dependency analysis capability to the Teradata MCP Server, comprising seven new tools for directed graph traversal across Teradata object lineage and data pipelines.

It also includes two infrastructure changes to app.py and utils/__init__.py that were developed as prerequisites for the graph tools but affect the broader server. These are called out explicitly below for maintainer review.

---

⚠️ Infrastructure Changes Requiring Maintainer Review

These two changes are not graph-specific. They affect the entire server and should be reviewed independently of the graph tool additions.

1. utils/__init__.py — create_response() returns dict instead of JSON string

Why: The MCP framework requires structured_content to be a dict (or None). The prior implementation returned a JSON string, which the framework wrapped in a [{"type": "text", ...}] list and rejected as malformed structured content. This caused silent response failures for tools that returned complex nested structures.

What changed:

• create_response() now returns a dict instead of calling json.dumps()
• New _make_serialisable() helper recursively converts all nested values to JSON-native Python types, ensuring None / SQL NULL values survive as None (JSON null) rather than the string "None"
• serialize_teradata_types() gains an explicit None guard for the same reason
• isinstance() calls updated to tuple form for Python 3.9 compatibility

Impact: Every tool that calls create_response() is affected. Existing tools that previously returned a JSON string will now return a dict — this is the correct behaviour per the MCP spec but is a breaking change for any caller that was parsing the JSON string directly.

2. app.py — get_tdconn() lazy factory removed; GRAPH_EDGE_CONTRACT resource registered

Why: The lazy get_tdconn() factory was a closure that deferred connection creation but added complexity with no clear benefit given that TDConn is already constructed eagerly at startup. Simplified to a direct tdconn = td.TDConn(settings=settings) call.

Note on registry system: The diff shows no registry system code in this branch. This is not a deletion — the registry system exists on the db-tool-registry branch and was never merged into main. There is nothing to restore.

Graph-specific addition: The graph://edge-contract MCP resource is registered when the graph_edge_contract resource pattern is present in the active profile. This serves the canonical Graph Edge Contract schema to AI agents so they understand the edge_repository parameter required by all graph tools.

---

New Capability: Graph Dependency Analysis Tools

Architecture

Seven tools covering the full graph analysis workflow, all stored-procedure-free. The only Teradata privilege required across the entire package is SELECT on an edge repository conforming to the Graph Edge Contract.

Step	Tool	Implementation	Purpose
0	graph_edgeContractDDL	Template	Generate edge repository DDL — no DB connection required
1	graph_findRootObjects	SQL	Discover objects with no upstream dependencies
2	graph_bfsLevels	Python BFS	Wave planning, deployment sequencing, blast-radius sizing
3	graph_traceLineage	Python + recursive CTE	Full lineage tracing, impact path analysis
4	graph_detectCycles	Python Union-Find + iterative DFS	Circular reference detection, DAG validation
5	graph_connectedComponents	Python Union-Find	Graph partitioning, isolated sub-graph identification
6	graph_analyseDatabase	Composite	All four analyses in one call with one shared edge fetch

Graph Edge Contract

All tools operate on an edge repository — any Teradata table or view exposing six required columns:

Src_Container_Name  Src_Object_Name  Src_Kind
Tgt_Container_Name  Tgt_Object_Name  Tgt_Kind

Plus two optional enrichment columns for visualisation clients:

Edge_Relationship   Transformation_Type

Edge direction is consistent across all edge types — Src is always upstream, Tgt always downstream — with three context-specific readings:

Edge type	Reading	Example
Object dependency	Src is referenced by Tgt	CUSTOMER_TABLE → CUSTOMER_VIEW
ETL input	Src is read by Tgt	CUSTOMER_TABLE → ETL_LOAD_JOB
ETL output	Src writes to Tgt	ETL_LOAD_JOB → CUSTOMER_FEATURES

The contract is served as an MCP resource at graph://edge-contract for agent discovery.

AI-Native Data Product Integration

The {ProductName}_Semantic.lineage_graph view (Observability Module v1.5 of the AI-Native Data Product standard) already conforms to this contract and can be used directly as edge_repository on any graph tool without generating DDL.

Progressive Disclosure Support

The package supports both MCP registration modes simultaneously:

• Static mode: graph_tools.py → GRAPH_TOOLS list → registration at startup
• Progressive Disclosure mode: __init__.py → ModuleLoader → ContextCatalog using docstrings

---

Supporting Infrastructure (Graph Wiring)

module_loader.py

Adds 'graph': 'teradata_mcp_server.tools.graph' to MODULE_MAP, enabling the ModuleLoader to discover and register graph tools when the graph prefix appears in a profile's tool patterns.

profiles.yml

Adds a graph profile entry:

graph:
  tool:
    - ^graph_.*
  prompt:
    - ^graph_.*
  resource:
    - ^graph_edge_contract$

---

Files Changed

Infrastructure (maintainer review requested)

File	Change
tools/utils/__init__.py	create_response() returns dict; _make_serialisable() added; NULL handling fixed
app.py	get_tdconn() simplified; graph://edge-contract resource registered; minor cleanups

Graph wiring

File	Change
tools/module_loader.py	graph prefix added to MODULE_MAP
config/profiles.yml	graph profile added

New graph package

File	Purpose
tools/graph/__init__.py	Re-exports all handle_* functions for ModuleLoader discovery
tools/graph/_graph_utils.py	Shared utilities: parse_csv_patterns, build_like_or, BFS helpers
tools/graph/graph_tools.py	Registration hub — all 7 tools in GRAPH_TOOLS list
tools/graph/graph_edge_contract.py	DDL generator + canonical GRAPH_EDGE_CONTRACT text (MCP resource)
tools/graph/graph_findRootObjects.py	Root object discovery
tools/graph/graph_bfsLevels.py	Pure-Python BFS
tools/graph/graph_traceLineage.py	Python + recursive CTE lineage analysis
tools/graph/graph_detectCycles.py	Python Union-Find + iterative DFS
tools/graph/graph_connectedComponents.py	Python Union-Find WCC
tools/graph/graph_analyseDatabase.py	Composite single-fetch analysis
tools/graph/README.md	Full tool reference documentation

---

Testing Notes

• All tools are SP-free — no stored procedure deployment required to test.
• graph_edgeContractDDL requires no database connection — DDL output can be verified without a Teradata instance.
• edge_repository validation can be tested by calling any graph tool without the parameter and confirming the error message and convention hint.
• The graph://edge-contract resource can be verified by fetching it via any MCP client with the graph profile active.
• The create_response() change should be regression-tested against existing tools (base, dba, sec) to confirm response structure is unchanged from the MCP client's perspective.