HEALER is a voice-first, multilingual agentic health companion that provides WHO-aligned emergency first-aid guidance and everyday symptom triage for 950M+ underserved Hindi and Urdu speakers, delivered through text, voice, and image input with full hands-free operation.
In the first 5–15 minutes after an incident — a roadside accident, a bad fall, a kitchen burn — outcomes often depend on immediate, basic actions. Yet many bystanders and families don't know what to do, freeze, or simply wait for an ambulance or a doctor.
This gap isn't about a lack of information online. It's that information isn't delivered hands-free, calmly, and step-by-step, in a way someone under stress, language pressure, or low literacy can actually follow. A wall of English text is not first aid.
The same gap shows up in everyday health too: a fever, a persistent headache, a rash that won't go away. People either don't know if it's serious, can't access a doctor quickly, or simply don't have anyone to ask in a language they're comfortable in.
For the hackathon MVP, HEALER focuses on bystanders responding to roadside or worksite accidents — the highest-stakes, most time-critical use case. The long-term vision extends to caregivers managing health at home and migrant workers navigating healthcare in a non-native language, where everyday triage (fever, colds, minor injuries) becomes just as central as emergency response.
HEALER delivers hands-free, multilingual guidance — voice in, voice out — that helps someone act immediately and safely:
- Voice-first interaction. Tap the mic and describe what happened, in English, Hindi, or Urdu.
- Optional image input. Upload a photo of an injury (e.g., a bleeding wound) to add context to the assessment.
- Agentic triage and red-flag awareness. Unlike a static FAQ, HEALER actively reasons about the specific situation described — identifying urgency cues and prioritizing escalation in real time, not retrieving a pre-written answer.
- Step-by-step spoken first aid. Clear, actionable guidance aligned to WHO first-aid principles: what to do now, what to avoid, and what to prepare until help arrives.
- Strong escalation and disclaimers. HEALER consistently encourages contacting emergency services for serious situations and is explicit that it's guidance support — not a diagnostic tool.
A bystander opens HEALER → taps the microphone → describes the situation (optionally uploading a photo) → HEALER triages for danger signs → HEALER speaks step-by-step stabilization guidance (bleeding control, positioning, safety checks) → HEALER prompts calling emergency services and provides a checklist to follow until help arrives.
The same engine, applied to a fever-and-headache query in Urdu or a burn in English, demonstrates HEALER's broader capability: calm, practical triage for everyday health — not just emergencies — in the user's own language.
HEALER is enabled by two shifts that have only recently become practical:
- Reasoning + multimodal AI that can support safer, structured, step-by-step guidance rather than generic Q&A.
- High-quality voice AI that makes hands-free, multilingual interaction practical in urgent, low-literacy contexts — making this kind of safety-critical assistant viable for the first time for underserved language communities.
HEALER is implemented as a Progressive Web App (PWA) with a Node.js/Express backend, powered by Microsoft AI Foundry and Phi-4 Reasoning, with:
- Azure Speech Services — voice input/output
- Azure Translator — multilingual support
- Azure Vision — optional image understanding
HEALER's emergency responses are grounded in publicly available WHO and established emergency-response guidance for bleeding control, choking, burns, and stabilization. (HEALER is not affiliated with or endorsed by WHO.)
HEALER does not diagnose conditions, prescribe medication, or replace a doctor, nurse, or emergency responder. For serious symptoms, injuries, or life-threatening emergencies, users should always contact emergency services or a qualified healthcare professional immediately. HEALER is a first step, not a final one.
By turning passive health information into immediate, spoken, step-by-step action, HEALER aims to reduce preventable harm in the critical minutes before professional care arrives — especially for communities where doctors are far away and language barriers are common.
git clone <repository-url>
cd healer
npm installCreate a .env file:
AZURE_AI_ENDPOINT=
AZURE_AI_KEY=
AZURE_TRANSLATOR_KEY=Run it:
npm run dev- Persistent health profiles — secure history tracking and emergency contact storage
- Session memory — follow-up care and context across conversations
- One-tap emergency calling, nearby hospital/pharmacy discovery, and location-aware recommendations
- More languages — Bengali, Tamil, Telugu, Marathi, Gujarati, Punjabi, and beyond
- Offline emergency mode for low-connectivity situations
Because in a medical emergency — or just a bad day — language, literacy, and access shouldn't be the things standing between you and help.
 |
 |
 |
 |
| Home Screen English | Home Screen Hindi | Home Screen Urdu | Hindi Chat Interface |