EchoSight

Contact-center call quality review tool — AI-powered moment detection, live agent simulation, and supervisor coaching dashboard.

EchoSight turns a raw call transcript into a fully annotated coaching report in a single API call. It combines deterministic rule-based moment detection (every flag is auditable) with Gemini-backed transcript simulation and a live interactive agent chat — all behind a clean REST API and an in-memory store that swaps to any database without touching business logic.

---

Quick Start

# 1. Install dependencies
cd echosight-backend
npm install

# 2. Set your Gemini API key
echo "GEMINI_API_KEY=your-key-here" > ../.env

# 3. Start the server
npm run dev        # nodemon watch mode
# or
npm start          # plain node

# Server listens at http://localhost:3001

POST /calls and GET /calls* work without a Gemini key.
/calls/simulate, /calls/simulate/stream, and all /chat/* routes require GEMINI_API_KEY.

---

System Architecture

graph TD
    Client(["Browser / Frontend Client"])

    subgraph Server["Express Server  :3001"]
        App["app.js\nCORS · JSON middleware\nStatic file serving"]
        CallsRouter["routes/calls.js\nTranscript ingestion\nAI simulation · SSE streaming"]
        ChatRouter["routes/chat.js\nLive session lifecycle\nMessage routing · Post-call summary"]
    end

    subgraph Services["Business Logic Layer"]
        GemSim["gemini-simulator.js\ngenerateTranscript()"]
        ChatAgent["chat-agent.js\ngetGreeting() · getChatResponse()\ngetFollowUp() · getChatSummary()"]
        MomDet["moment-detector.js\ndetectMoments()"]
        Summ["summarizer.js\ncomputeSummary()"]
    end

    subgraph Storage["Storage Layer"]
        MemDB["memory-db.js\nsaveCall · getCall · listCalls · getMoments"]
        ChatSess["chat-sessions.js\ncreateSession · getSession · deleteSession"]
    end

    Gemini(["Gemini 2.5 Flash API"])

    Client -->|"REST / SSE"| App
    App --> CallsRouter
    App --> ChatRouter
    CallsRouter --> GemSim
    CallsRouter --> MomDet
    CallsRouter --> Summ
    CallsRouter --> MemDB
    ChatRouter --> ChatAgent
    ChatRouter --> MomDet
    ChatRouter --> Summ
    ChatRouter --> MemDB
    ChatRouter --> ChatSess
    GemSim --> Gemini
    ChatAgent --> Gemini

Loading

Element	Description
Browser / Frontend Client	Any HTTP client — browser, Postman, or a React frontend hitting localhost:3001
app.js	Express root. Registers CORS (wildcard), JSON body parsing, mounts both routers, serves index.html for all non-API routes
routes/calls.js	All /calls/* endpoints: ingestion, AI simulation (one-shot + SSE), listing, and moment retrieval
routes/chat.js	All /chat/* endpoints: session start, per-message exchange, and session end with AI coaching summary
gemini-simulator.js	Generates realistic call transcripts from a one-line issue description. Self-validates output against moment rules and retries if any type is missing
chat-agent.js	Manages the live Gemini conversation: greeting, per-message response, automatic follow-up when agent says "let me check", post-call coaching summary
moment-detector.js	Stateless, deterministic rule engine. Scans turns for escalation keywords, empathy phrases, dead-air gaps, and long monologues. Zero ML
summarizer.js	Computes the full analytics payload: talk time, response timing, word stats, sentiment arc, empathy score, quality score
memory-db.js	In-memory Map store. Exports exactly four functions (saveCall, getCall, listCalls, getMoments). Swap this file for a DB adapter — nothing else changes
chat-sessions.js	In-memory store for live sessions. Holds Gemini conversation history and turn buffer for the duration of one call
Gemini 2.5 Flash API	Powers transcript generation (JSON mode), live agent conversation, and post-call coaching analysis

---

Call Ingestion Flow

sequenceDiagram
    participant C as Client
    participant R as routes/calls.js
    participant MD as moment-detector.js
    participant S as summarizer.js
    participant DB as memory-db.js

    C->>R: POST /calls { callId, agentName, duration, turns[] }
    R->>MD: detectMoments(turns)
    MD-->>R: moments[]
    R->>S: computeSummary(turns, moments, duration)
    S-->>R: summary { callQualityScore, talkTimeSeconds,\navgAgentResponseTime, longestTurn, … }
    R->>DB: saveCall({ callId, agentName, duration, turns, moments, summary })
    R-->>C: 201 { callId, momentCount }

Loading

Element	Description
POST /calls	Entry point for external transcript ingestion. Requires callId, agentName, turns[]. duration is optional but enables accurate time-based talk ratios
detectMoments(turns)	Stateless scan — no DB reads. Returns one moment object per rule hit. A single turn can produce multiple moments simultaneously
computeSummary(turns, moments, duration)	Pure function, no side effects. duration is used to approximate last-turn speaking time for talkTimeSeconds
saveCall(...)	Persists the complete call object keyed by callId. Returns 409 if callId already exists
201 { callId, momentCount }	Lightweight acknowledgement. Full annotated data is retrievable via GET /calls/:id

---

AI Simulation Flow (SSE Streaming)

sequenceDiagram
    participant C as Client
    participant R as routes/calls.js
    participant G as gemini-simulator.js
    participant API as Gemini API
    participant MD as moment-detector.js
    participant DB as memory-db.js

    C->>R: GET /calls/simulate/stream?issue=billing+dispute
    R-->>C: SSE headers (text/event-stream)
    R-->>C: event:status  { message: "AI is generating…" }
    R->>G: generateTranscript(issue, agentName)
    G->>API: generateContent (JSON mode, T=0.7)
    API-->>G: raw transcript JSON
    G->>MD: detectMoments(turns) — self-validate
    alt missing moment types
        G->>API: retry with targeted fix hints (T=0.5)
        API-->>G: corrected transcript
    end
    G-->>R: { agentName, duration, turns[] }
    R-->>C: event:start  { agentName, duration, turnCount }
    loop each turn — scaled delay 400ms–2500ms
        R-->>C: event:turn  { index, speaker, text, t, moments[] }
    end
    R->>DB: saveCall(...)
    R-->>C: event:complete  { callId, summary, momentCount }

Loading

Element	Description
GET /calls/simulate/stream	Opens an SSE connection. Requires issue query param. Optional agentName (default: Priya)
SSE headers	Sets Content-Type: text/event-stream, Cache-Control: no-cache, X-Accel-Buffering: no to prevent proxy buffering
event:status	First event fires immediately, before Gemini latency, so the client knows something is happening
generateTranscript()	Sends a structured system prompt with 4 hard rules (≥2 escalations, ≥2 empathy phrases, exactly one dead-air gap >15s, ≥1 monologue >50 words)
self-validate	detectMoments() runs on the AI's own output. If any moment type is absent, a targeted retry prompt lists exactly what to fix
scaled delay	Per-turn delay = gap_seconds × 1000 × 0.08, clamped 400ms–2500ms. Gives a realistic replay feel without wasting real time
event:turn	Each turn is emitted individually with its detected moments already attached — the client can highlight moments as they arrive
event:complete	Final event. callId is assigned after streaming completes. Contains the full summary object

---

Live Chat Session Flow

sequenceDiagram
    participant U as User (Customer)
    participant R as routes/chat.js
    participant CA as chat-agent.js
    participant API as Gemini API
    participant MD as moment-detector.js
    participant DB as memory-db.js

    U->>R: POST /chat/start { agentName }
    R->>CA: getGreeting(agentName)
    CA->>API: system prompt + "BEGIN_CALL"  T=0.9
    API-->>CA: greeting text
    R-->>U: { sessionId, agentMessage, t:0, moments:[] }

    loop Each message exchange
        U->>R: POST /chat/:id/message { text }
        Note over R: Record customer turn + timestamp
        R->>MD: detectMoments(session.turns)
        R->>CA: getChatResponse(geminiHistory, agentName)  T=0.8
        CA->>API: full conversation history
        API-->>CA: agent reply text
        alt agent said "let me check / one moment / bear with me…"
            R->>CA: getFollowUp(geminiHistory, agentName)  T=0.7
            CA->>API: SYSTEM follow-up trigger
            API-->>CA: results / next steps
            Note over R: Timestamp set to agentT+4s to avoid false dead_air
        end
        R-->>U: { agentMessage, agentT, customerMoments,\nagentMoments, followUp, allMoments }
    end

    U->>R: POST /chat/:id/end
    R->>CA: getChatSummary(turns, agentName)
    CA->>API: transcript + analysis prompt  JSON mode T=0.3
    API-->>CA: { conclusion, resolved, customerSatisfied,\nagentPerformance, coachingNote }
    R->>DB: saveCall(...)
    R-->>U: { callId, duration, summary, aiSummary }

Loading

Element	Description
POST /chat/start	Creates session in chat-sessions.js, calls Gemini for a greeting, returns sessionId used for all subsequent calls
getGreeting()	Temperature 0.9 — higher randomness for a natural-sounding opening. Sends BEGIN_CALL trigger to the agent persona system prompt
session.geminiHistory	Running [{ role, parts }] array passed to every Gemini call. Maintains full conversation context across the entire session
getChatResponse()	Temperature 0.8. Sends full geminiHistory so Gemini has context of everything said so far
needsFollowUp()	Checks agent reply against 15 trigger phrases ("let me check", "one moment", "bear with me", etc.)
getFollowUp()	Appends a SYSTEM message instructing the model to deliver lookup results. Timestamp set to agentT + 4 to avoid triggering a false dead_air moment between the two consecutive agent turns
getChatSummary()	Separate Gemini call in JSON mode at temperature 0.3 (low for consistent structured output). Returns 5 fields the UI can display as a coaching card
POST /chat/:id/end	Computes final moments + summary, calls getChatSummary(), saves to memory-db, deletes the live session

---

Moment Detection Rules

flowchart TD
    IN["Turn { speaker, text, t }"]

    IN --> E1{speaker === 'customer'\nAND text contains\nescalation keyword?}
    E1 -- yes --> EM1["🔴 escalation_signal\nkeywords: cancel · refund · manager\nlawsuit · ridiculous · unacceptable"]

    IN --> E2{speaker === 'agent'\nAND text contains\nempathy phrase?}
    E2 -- yes --> EM2["🟢 empathy_statement\nphrases: 'I understand' · 'I'm sorry'\n'I apologise' · 'I can see why'"]

    IN --> E3{speaker === 'agent'\nAND prev turn === 'customer'\nAND gap > 15s?}
    E3 -- yes --> EM3["🟡 dead_air\nmatchedText: 'Xs gap'\nonly fires on agent response turns"]

    IN --> E4{word count > 50?}
    E4 -- yes --> EM4["🔵 long_monologue\nmatchedText: 'N words'\napplies to either speaker"]

    EM1 & EM2 & EM3 & EM4 --> OUT["moment[]\ntype · turnIndex · t · speaker · matchedText"]

Loading

Element	Description
Turn input	A single element of turns[]. Required fields: speaker (agent|customer), text, t (seconds from call start)
escalation_signal	Fires on customer turns only. 6 keywords chosen for churn and conflict signals
empathy_statement	Fires on agent turns only. 4 exact phrases — the same ones the Gemini agent persona is instructed to produce, closing the validation loop
dead_air	Fires on the agent's response turn when gap from the previous customer turn exceeds 15 seconds. Measures company-side delay only
long_monologue	Fires on any turn exceeding 50 words. Catches customer venting or agent over-explaining a resolution
moment output	Each object carries turnIndex for O(1) annotation lookup, t for timeline display, and matchedText for supervisor audit

---

Call Quality Score

flowchart LR
    Base["base = 100"]

    Base --> A["− 10 per unaddressed escalation\nmax(0, escalations − empathy responses)"]
    Base --> B["− 15 per dead_air instance"]
    Base --> C{sentimentArc}

    C -->|improved| D["+ 10"]
    C -->|declined| E["− 15"]
    C -->|neutral| F["± 0"]

    A & B & D & E & F --> Clamp["clamp to 0–100"]
    Clamp --> Score["callQualityScore  0–100"]

Loading

Element	Description
base = 100	Every call starts at a perfect score. Points are deducted for measurable failures only
unaddressed escalation −10	An escalation that was not followed by any empathy phrase. Each one costs 10 points
dead_air −15	Each dead-air instance costs 15 points — weighted higher than a missed empathy phrase because it is a harder, more objective failure
sentimentArc +10 / −15	Computed by comparing escalation density in first vs. second half. improved means the agent de-escalated; declined means it got worse. Asymmetric weights (−15 > +10) because decline is a stronger signal
clamp 0–100	Prevents extreme multi-failure calls from producing negative scores

---

Summary Statistics Reference

Every call returns a summary object with the following fields:

mindmap
  root((summary))
    Turn Counts
      totalTurns
      agentTurnCount
      customerTurnCount
    Talk Ratio
      talkRatio.agent %
      talkRatio.customer %
    Talk Time
      talkTimeSeconds.agent
      talkTimeSeconds.customer
      talkTimeSeconds.agentPct
      talkTimeSeconds.customerPct
    Word Stats
      avgTurnWords.agent
      avgTurnWords.customer
      avgTurnWords.overall
      longestTurn.speaker
      longestTurn.words
      longestTurn.preview
    Response Timing
      avgAgentResponseTime
      maxAgentResponseTime
      longestSilence
      firstResponseTime
    Quality Metrics
      callQualityScore
      escalationCount
      empathyScore
      empathyCount
      agentEmpathyRate
      deadAirCount
      sentimentArc
      momentBreakdown

Loading

Field	Type	Description
totalTurns	number	Total turn count across all speakers
agentTurnCount	number	Turns where speaker === 'agent'
customerTurnCount	number	Turns where speaker === 'customer'
talkRatio	{ agent, customer }	Percentage share of turns per speaker (turn-count based)
talkTimeSeconds	{ agent, customer, agentPct, customerPct }	Approximate speaking seconds derived from timestamp gaps. More accurate than talkRatio for calls with lopsided turn lengths
avgTurnWords	{ agent, customer, overall }	Mean word count per turn, broken out by speaker
longestTurn	{ speaker, words, turnIndex, t, preview }	Turn with the highest word count. preview is the first 80 characters
avgAgentResponseTime	number (s)	Mean gap on all customer→agent transitions. Core measure of agent responsiveness
maxAgentResponseTime	number (s)	Worst single agent response delay in the call
longestSilence	number (s)	Largest gap between any two consecutive turns (not filtered to customer→agent)
firstResponseTime	number (s)	Timestamp of the first agent turn — how quickly the agent opened the call
callQualityScore	number 0–100	Composite quality score (see formula above)
escalationCount	number	Total escalation_signal moments detected
empathyScore	number 0–1.0	min(1, empathyCount / escalationCount). 1.0 = every escalation was addressed
empathyCount	number	Total empathy_statement moments detected
agentEmpathyRate	number %	Percentage of agent's turns that contained an empathy phrase
deadAirCount	number	Total dead_air moments (>15s gaps on agent response)
sentimentArc	improved | neutral | declined	Call trajectory based on escalation distribution across first vs. second half
momentBreakdown	{ escalation_signal, empathy_statement, dead_air, long_monologue }	Count of each moment type

---

Data Model

erDiagram
    CALL {
        string callId PK
        string agentName
        number duration
    }
    TURN {
        string speaker
        string text
        number t
    }
    MOMENT {
        string type
        number turnIndex
        number t
        string speaker
        string matchedText
    }
    SUMMARY {
        number totalTurns
        number callQualityScore
        number escalationCount
        number empathyScore
        number avgAgentResponseTime
        number longestSilence
        string sentimentArc
        object talkTimeSeconds
        object avgTurnWords
        object longestTurn
    }

    CALL ||--o{ TURN : "has"
    CALL ||--o{ MOMENT : "has"
    CALL ||--|| SUMMARY : "has"

Loading

Entity	Field	Description
CALL	callId	Primary key. Format: call-001 (ingested) or sim-{timestamp} (simulated) or call-{timestamp} (chat)
CALL	agentName	Agent display name used in Gemini persona prompts
CALL	duration	Total call length in seconds. Optional on ingestion; computed from startTime on chat sessions
TURN	speaker	agent or customer. Determines which detection rules apply
TURN	text	Raw spoken text. No formatting — plain conversational speech
TURN	t	Cumulative seconds from call start. Used for gap calculations and timeline display
MOMENT	type	One of: escalation_signal, empathy_statement, dead_air, long_monologue
MOMENT	turnIndex	Index into turns[] — enables O(1) annotation lookup when building annotatedTurns
MOMENT	matchedText	The exact keyword, phrase, or measurement that triggered the rule. Used for supervisor audit
SUMMARY	talkTimeSeconds	Object containing agent, customer, agentPct, customerPct — time-based talk distribution
SUMMARY	longestTurn	Object: { speaker, words, turnIndex, t, preview } — the most verbose turn in the call

---

Project Structure

EchoSight/
├── .env                          # GEMINI_API_KEY (git-ignored)
├── index.html                    # Static dashboard served at /
├── README.md                     # This file
├── ECHOSIGHT_DOCS.md             # Full technical reference
└── echosight-backend/
    ├── index.js                  # Entry point — starts Express on :3001
    ├── package.json
    └── src/
        ├── app.js                # Express app, middleware, route mounts, static serving
        ├── routes/
        │   ├── calls.js          # /calls/* — ingestion, simulation, SSE, listing
        │   └── chat.js           # /chat/* — session start, message, end
        ├── services/
        │   ├── moment-detector.js  # Deterministic rule engine (no ML)
        │   ├── summarizer.js       # Analytics computation — all summary fields
        │   ├── gemini-simulator.js # AI transcript generation with self-validation
        │   └── chat-agent.js       # Live AI agent: greeting, response, follow-up, summary
        └── storage/
            ├── memory-db.js        # Call store — swap this file to change DB
            └── chat-sessions.js    # Live session store

---

Key Design Decisions

Why deterministic detection instead of LLM classification?
Every moment flag is traceable to a specific keyword or timestamp. Supervisors can audit findings. LLM classifiers produce scores — not reasons. For compliance review and agent feedback, auditability beats marginal accuracy.

Why does the simulator validate its own output?
Gemini occasionally misses required moment types despite explicit instructions. Running detectMoments() on the output before returning it makes the simulator and the detector share the same truth. If the check fails, a targeted retry prompt tells the model exactly which types are missing.

Why is talkTimeSeconds separate from talkRatio?
talkRatio counts turns. A 2-word reply and a 200-word explanation both count as 1 turn. talkTimeSeconds uses the timestamp gap to the next turn as a proxy for speaking duration — a far more meaningful measure of who dominated the conversation.

Why is storage a 4-function contract in its own file?
Every route and service goes through memory-db.js. Swapping to Postgres, Redis, or MongoDB means rewriting one file with the same four function signatures. No other file is aware that storage exists.

---

License

Apache 2.0 — see echosight-backend/package.json.