PR extractAI 1st review

Author: magmacomputing

packages/plugins/ai/CHANGELOG.md

@@ -35,7 +35,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Cascading Cache TTL Hierarchy**: Implemented strict 4-tier TTL resolution (`options.ttl` > `provider.ttl` > `global config.ttl` > default 1 hour / 3,600,000 ms for `parseAI` or 24 hours / 86,400,000 ms for context/difference handlers) for fine-grained cache entry expiration control on stores enforcing TTL.
 - **Fail-Open Storage Resilience**: Custom cache adapter errors during `get()` or `set()` operations fail open silently to direct LLM resolution, preserving application request uptime.
 - **Dynamic Remote Provider Manifest (`loadRemoteManifest`)**: Lazily fetches remote provider defaults (`providers.v1.json`) on initialization with a 1500ms timeout and automatic air-gapped fallback to compiled `DEFAULT_PROVIDERS`.
-- **Cache Eviction Synchronization**: Enhanced `clearAiCache()` to flush matching keys and prefixes across both `Tempo.cache` and custom `AiCacheAdapter` storage engines, returning `Promise<void>` to await async adapter eviction.
+- **Cache Eviction Synchronization**: Enhanced `aiCache.clear()` to flush matching keys and prefixes across both `Tempo.cache` and custom `AiCacheAdapter` storage engines, returning `Promise<void>` to await async adapter eviction.
 
 ### Changed & Hardened
 - **Consensus Mode TTL Resolution**: Fixed a runtime bug where standard provider TTL lookups failed in Consensus mode due to the synthetic sentinel provider ID (`'consensus'`), which caused lookups on the winning provider array to return undefined. Now reduces over all participating provider configs to select the minimum (most conservative) TTL.
@@ -65,11 +65,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Non-Destructive Glossary Appending**: Custom glossaries provided via `initAI({ cache })` are safely appended to `Tempo.cache` as static immortal terms without destructive overrides.
 - **Silent Native Pre-Parsing & Cache Controls**: `parseAI` attempts fast, zero-latency native `Tempo` resolution and checks `Tempo.cache` before initiating LLM network calls. Supports `cache: false` to bypass cache lookups and `force: true` to force a fresh LLM API request.
 - **Anchor Instance Reuse & Cache Salting**: Reuses anchor `Tempo` instances to minimize memory allocations and salts cache keys with the anchor's date and system context (`timeZone`, `calendar`, `locale`, `sphere`), preventing stale cache hits across midnight boundaries or context shifts.
-- **Resilient Cache Invalidation**: Normalized cache key input (whitespace trimming and case insensitivity) for `clearAiCache` and internal lookups.
+- **Resilient Cache Invalidation**: Normalized cache key input (whitespace trimming and case insensitivity) for `aiCache.clear()` and internal lookups.
 
 ## [0.1.0] - 2026-07-26
 
 ### Added
 - Initial scaffolding of the AI natural language parsing plugin.
-- Functional exports for `parseAI`, `initAI`, and `clearAiCache`.
+- Functional exports for `parseAI`, `initAI`, and `aiCache`.
 - Initial provider fallback-routing engine supporting HTTP requests to configured LLM provider endpoints.

packages/plugins/ai/doc/architecture.md

@@ -197,6 +197,9 @@ export async function POST(req: Request, env?: { GROQ_API_KEY?: string }) {
 
 ## 🔒 Security & Privacy Guarantees
 
+> [!TIP]
+> For an in-depth breakdown of our automated PII redaction, Smart Debug infrastructure, and tamper-resistant Proxy introspection, see the dedicated **[Security & Privacy Architecture Guide (`security.md`)](./security.md)**.
+
 Whether running directly on backend servers (Node.js, Deno, Bun, Edge Workers) or through a client-side browser proxy, `@magmacomputing/tempo-plugin-ai` enforces strict security and privacy standards:
 
 ### 1. Transport Security (HTTPS / TLS)
@@ -207,10 +210,10 @@ Temporal processing payloads (dates, times, context snippets, prompts) are proce
 
 ### 3. In-Memory Credential Redaction & Immutability
 * **Automated Key Redaction**: Calling `getAiConfig()` returns a sanitized, deeply read-only snapshot of active configurations with all provider `key` values replaced with `[REDACTED]`, preventing accidental exposure in log files, APM traces, or crash dumps.
-* **Frozen Metadata**: All diagnostic metadata attached to `Tempo` instances via `.ai` is deeply frozen with `Object.freeze()` and guarded via runtime `Proxy` wrappers, protecting against direct runtime mutation of the `.ai` metadata.
+* **Frozen Metadata**: All diagnostic metadata attached to `Tempo` instances via `.ai` and all structured AI result objects (`TempoAiFormatResult`, `TempoAiExtractResult`, `TempoAiDiffResult`, `TempoScheduleResult`, `TempoRecurrenceResult`, `TempoContext`) are deeply frozen with `Object.freeze()` and guarded via runtime `Proxy` wrappers, protecting against direct runtime mutation.
 
 ### 4. Deterministic Schema Guardrails & Confidence Range Verification
-All LLM prompts are paired with rigid, machine-verifiable JSON schemas. Responses undergo strict boundary validation, regex parsing, confidence range verification (`0.0` to `1.0`), and ISO verification before any native `Tempo` date object or result payload is instantiated. If an LLM returns malformed, out-of-range, or non-chronological data, the plugin throws a typed `TempoAiError` or triggers automatic fallback rather than silently returning an invalid date.
+All LLM prompts are paired with rigid, machine-verifiable JSON schemas. Responses undergo strict boundary validation, regex parsing, confidence range verification (`0.0` to `1.0`), and schema verification before any native `Tempo` date object or structured result payload is instantiated. If an LLM returns malformed, out-of-range, or unparseable data, the plugin throws a typed `TempoAiError` or triggers automatic provider fallback rather than silently propagating corrupt data.
 
 ### 5. Partitioned Caching & Fail-Open Storage Resilience
 * **Strict Cache Key Partitioning**: Caches are namespaced and hashed (`ai:<namespace>::...`) with timezone, locale, calendar, and anchor date isolation to prevent cross-tenant or cross-regional cache poisoning.

packages/plugins/ai/doc/context.md

@@ -2,6 +2,9 @@
 
 `contextAI()` is designed to analyze unstructured or ambiguous text (e.g. user biographies, locations, or email bodies) and infer their regional configuration settings. It resolves these properties to a standard configuration object containing timezone, locale, calendar system, and hemisphere.
 
+> [!TIP]
+> **Smart Debug Telemetry**: Enabling `debug: true` activates operational logs. In production environments (`NODE_ENV === 'production'`), PII (emails, phone numbers, auth tokens) is automatically sanitized and masked in console output and terminal inspections. See the [Security & Privacy Architecture Guide](./security.md).
+
 This is highly useful for user onboarding settings, automatic context mapping for calendar syncs, and geolocating inputs dynamically without maintaining static geographic mapping tables.
 
 ---
@@ -28,7 +31,7 @@ console.log(context.timeZone);  // "Australia/Sydney"
 console.log(context.locale);    // "en-AU"
 console.log(context.calendar);  // "gregory"
 console.log(context.sphere);    // "south"
-console.log(context.confidence); // 0.98
+console.log(context.confidence);// 0.98
 ```
 
 ---
@@ -46,7 +49,7 @@ console.log(context.confidence); // 0.98
 | **`providers`** | `AiProvider[]` | Per-request provider configuration overrides. |
 | **`timeout`** | `number` | Per-request timeout in milliseconds (overrides provider and global timeouts). |
 | **`hedgeDelay`** | `number` | Delay in milliseconds before initiating speculative hedging in `AiMode.Hedged` (default: `800ms`). |
-| **`debug`** | `boolean` | If true, logs prompt context and LLM payloads to console. |
+| **`debug`** | `boolean` | If true, logs prompt context and cache operations to console (automatically PII-sanitized in production). |
 | **`softErrors`** | `boolean` | If true, returns `TempoAiError` into array index position instead of rejecting the entire batch query. |
 
 ---

packages/plugins/ai/doc/grounding.md

@@ -37,9 +37,13 @@ Passing the `Locale` and `TimeZone` is critical for the LLM to know whether `"11
 
 ## The Decoupled Output Bridge
 
-To ensure deterministic behavior, the LLM is instructed to *only* return strict ISO 8601 strings. 
+To ensure deterministic, type-safe behavior, the plugin enforces a strict decoupled bridge between AI text generation and JavaScript object hydration:
 
-The plugin executes the network request, the LLM returns a local ISO string without a timezone offset or 'Z' suffix (like `"2026-11-26T00:00:00"`), and the plugin immediately passes that string back into the native `new Tempo()` constructor. The provider response must omit timezone suffixes to match the local ISO contract enforced by Tempo AI functions. The developer seamlessly receives a valid, native `Tempo` instance. This eliminates AST-construction ambiguity, creating a decoupled bridge between AI text generation and native Tempo conversion.
+* **For Point-in-Time Parsing (`parseAI`)**: The LLM is instructed to return a strict local ISO 8601 string without a timezone offset or 'Z' suffix (e.g. `"2026-11-26T00:00:00"`). The plugin immediately constructs a native `new Tempo()` instance with caller-defined timezone and calendar context.
+* **For Structured Functions (`formatAI`, `extractAI`, `diffAI`, `contextAI`)**: The LLM completes rigid JSON schemas validated against strict boundary rules, instantiating typed result objects (`TempoAiFormatResult`, `TempoAiExtractResult`, `TempoAiDiffResult`, `TempoContext`).
+* **For Intervals & Generators (`scheduleAI`, `recurrenceAI`)**: The plugin hydrates interval boundaries into a proxied `Interval<Tempo>` or exposes an iterable generator yielding sequential `Tempo` instances.
+
+This eliminates AST-construction ambiguity and provides clean runtime contracts for every operation.
 
 ### Relative Date Ambiguity Tie-Breakers
 
@@ -48,11 +52,13 @@ To eliminate model variance on idioms like "Next Friday" or "Last Tuesday", the
 * `"last [weekday/unit]"` / `"previous [weekday/unit]"`: Evaluated as the most recent past occurrence prior to the grounding anchor.
 * `"this [weekday]"`: Evaluated as the occurrence within the current calendar week containing the grounding anchor.
 
-### Confidence Thresholds & Metadata (`.ai`)
+### Confidence Thresholds & Metadata Handling
 
-When `minConfidence` is supplied in options (e.g. `parseAI("...", { minConfidence: 0.8 })`), any LLM response returning a confidence score below that threshold produces a `Tempo` instance with `isValid === false`. 
+When `minConfidence` is supplied in options (e.g. `{ minConfidence: 0.85 }`):
+* **`parseAI`**: Any LLM response returning a confidence score below the threshold produces a `Tempo` instance with `isValid === false` (when using `softErrors: true`) or throws a `TempoAiError(422)`.
+* **Structured Functions (`formatAI`, `extractAI`, `diffAI`, `contextAI`, `scheduleAI`, `recurrenceAI`)**: Low-confidence completions immediately throw a `TempoAiError(422)` (or return a `TempoAiError` in batch arrays when `softErrors: true` is enabled).
 
-Every resolved `Tempo` instance returned by `parseAI` (and other AI functions) has a non-writable, frozen `.ai` metadata descriptor containing execution audit data:
+Every resolved `Tempo` instance returned by `parseAI` has a non-writable, frozen `.ai` metadata descriptor containing execution audit data:
 ```typescript
 const dt = await parseAI("Christmas 2026", { debug: true });
 console.log(dt.ai);
@@ -67,3 +73,5 @@ console.log(dt.ai);
 //   normalizedPrompt: 'christmas 2026' // Present when debug is enabled
 // }
 ```
+
+*(For other AI functions like `extractAI` or `diffAI`, diagnostic metadata including `confidence`, `reasoning`, and `provider` is attached directly to the returned result object.)*