🧠 Emotion intelligence meets clinical data — production-ready NLP sentiment analysis pipeline powered by DistilRoBERTa + structured medical report extraction via Pytesseract OCR + intelligent cross-referencing against a curated medical conditions database — served through a blazing-fast Flask REST API.

---

📡 System Architecture

╔══════════════════════════════════════════════════════════════════════════╗
║                        🧠  MindScan AI  System                          ║
╠════════════════════════╦══════════════════════════╦═══════════════════════╣
║   💬 NLP ENGINE        ║   📄 OCR PIPELINE        ║   🔗 CROSS-REF        ║
║                        ║                          ║                       ║
║  User Text Input       ║  Medical Image Upload    ║  Emotion JSON  +      ║
║        ↓               ║          ↓               ║  OCR Report JSON      ║
║  DistilRoBERTa         ║  Pytesseract + PIL       ║          ↓            ║
║  (7-class emotions)    ║  Grayscale → PSM 6       ║  medical_keywords     ║
║        ↓               ║          ↓               ║  .json  DB  Lookup    ║
║  Confidence Score      ║  Regex Parser            ║          ↓            ║
║  + Risk Assessment     ║  vitals · labs · meds    ║  Correlation Engine   ║
║        ↓               ║          ↓               ║  + Matched Conditions ║
║  Personalised Advice   ║  Structured JSON Output  ║                       ║
╚════════════════════════╩══════════════════════════╩═══════════════════════╝
                                     ↓
               ┌──────────────────────────────────────────┐
               │   🐍  Flask REST API  ·  port 5000        │
               │   CORS-enabled  ·  JSON responses         │
               └──────────────────────────────────────────┘
                                     ↓
               ┌──────────────────────────────────────────┐
               │   🌐  Vanilla JS SPA  ·  Dark Mode UI     │
               │   Drag & Drop  ·  Emotion Bars  ·  OCR   │
               └──────────────────────────────────────────┘

---

✨ Feature Matrix

Feature	Technology	Details	Status
💬 7-Class Emotion NLP	distilroberta-base	joy · sadness · anger · fear · disgust · surprise · neutral	✅ Live
📊 Confidence Scoring + Risk Levels	EMOTION_MENTAL_MAP ruleset	low · low-medium · medium · high	✅ Live
🏥 Medical Image OCR	pytesseract + Pillow	Grayscale enhance · PSM 6 config	✅ Live
🧪 Lab Value Extraction	Regex NLP pipeline	HbA1c · Glucose · WBC · RBC · TSH · T3 · T4	✅ Live
❤️ Vitals Detection	Regex pattern matching	BP · HR · SpO2 · BMI · Temp · Weight	✅ Live
💊 Medication Parsing	Dosage unit detection	mg · mcg · ml lines extracted	✅ Live
🔗 Emotion × Medical Cross-Ref	JSON rule engine	Condition–emotion clinical correlation	✅ Live
🌐 Dark Mode SPA	Vanilla HTML/CSS/JS	Drag & drop · animated bars · typing UX	✅ Live
🔒 Pre-commit Security Pipeline	flake8 + codespell + secret scan	PEP8 enforced · secrets blocked	✅ Live

---

🧠 Emotion Classification Engine

Model: j-hartmann/emotion-english-distilroberta-base · 124M parameters · fine-tuned across 6 emotion datasets

Emotion	Risk Level	Color Token	Clinical Recommendation
😊 joy	🟢 low	green	Reinforce positive behavioural routines
😢 sadness	🟡 medium	blue	Encourage therapeutic support & social connection
😠 anger	🟡 medium	orange	Deep breathing · CBT journaling · cool-down protocol
😨 fear	🔴 high	red	Grounding techniques · immediate professional referral
😒 disgust	🟠 low-medium	yellow	Boundary reflection · counselling recommended
😲 surprise	🟢 low	teal	Allow cognitive processing time for new stimuli
😐 neutral	🟢 low	gray	Maintain consistent self-care & wellness practices

Note

The UNEXPECTED key warning for roberta.embeddings.position_ids at model load is harmless — known cross-architecture artifact, safely ignored.

---

📁 Project Structure

AI-Mental-Health-Chatbot-OCR-Scanner/
│
├── 🐍 backend/
│   ├── app.py                  ←  Flask REST API server · routes + CORS
│   ├── sentiment_model.py      ←  DistilRoBERTa NLP emotion classifier
│   ├── ocr_pipeline.py         ←  Pytesseract OCR + regex report parser
│   ├── medical_crossref.py     ←  Emotion × medical DB cross-referencer
│   └── requirements.txt        ←  Pinned Python dependencies
│
├── 📦 data/
│   └── medical_keywords.json   ←  Curated conditions · symptoms · medications DB
│
├── 🌐 frontend/
│   └── index.html              ←  SPA dark-mode UI · vanilla JS
│
├── ⚙️  scripts/                ←  Automation + helper scripts
├── 🔧 .github/                 ←  GitHub Actions CI/CD workflows
├── 🪝 .pre-commit-config.yaml  ←  flake8 · secret scan · codespell hooks
└── 📋 requirements.txt         ←  Root-level requirements

---

🚀 Quick Start

1️⃣ Prerequisites

Platform	Install Tesseract OCR
🪟 Windows	UB-Mannheim installer → add to PATH
🍎 macOS	brew install tesseract
🐧 Linux	sudo apt install tesseract-ocr

2️⃣ Clone & Install

git clone https://github.com/your-username/AI-Mental-Health-Chatbot-OCR-Scanner.git
cd AI-Mental-Health-Chatbot-OCR-Scanner
python -m venv venv
venv\Scripts\activate        # Windows
source venv/bin/activate     # macOS / Linux
cd backend && pip install -r requirements.txt

3️⃣ Launch

python app.py
# → http://127.0.0.1:5000

---

🔌 API Endpoint Reference

---

💬 Emotion Analysis — NLP Operations

Analyze free-text for emotional state using transformer-based NLP classification Field Value Method URL http://localhost:5000/api/chat Content-Type application/json Returns Dominant emotion · confidence · risk level · AI advice curl -X POST http://localhost:5000/api/chat \ -H "Content-Type: application/json" \ -d '{"message":"I have been feeling really anxious and overwhelmed lately."}' 📤 Sample Response · 200 OK { "status": "success", "emotion_analysis": { "dominant_emotion": "fear", "confidence": 78.4, "risk_level": "high", "color": "red", "advice": "Fear and anxiety are linked. Grounding techniques and professional support are recommended.", "all_emotions": [ { "label": "fear", "score": 78.4 }, { "label": "sadness", "score": 12.1 }, { "label": "neutral", "score": 5.3 }, { "label": "anger", "score": 2.8 } ] } } Response Schema Field Type Description dominant_emotion string Highest-confidence predicted emotion label confidence float Probability score 0–100 for dominant class risk_level string low · low-medium · medium · high color string UI color token mapped to emotion advice string Personalised AI-generated mental health guidance all_emotions array Full softmax probability distribution (sorted)   Missing or empty message field

---

📄 Medical Report Scanner — OCR Operations

Upload a medical report image — extracts vitals, lab values, medications and diagnoses via computer vision OCR Field Value Method URL http://localhost:5000/api/ocr Content-Type multipart/form-data Returns Structured patient data · vitals · labs · medications · diagnoses curl -X POST http://localhost:5000/api/ocr \ -F "file=@/path/to/medical_report.png" 📤 Sample Response · 200 OK { "status": "success", "ocr_result": { "patient_name": "John Doe", "date": "10/03/2026", "vitals": { "BP": "120/80", "HR": "72 bpm", "SpO2": "98%", "BMI": "22.4" }, "lab_values": { "Glucose": "95 mg/dL", "HbA1c": "5.4%", "Cholesterol": "182 mg/dL", "Hemoglobin": "13.8 g/dL" }, "diagnoses": ["Impression: Generalized Anxiety Disorder"], "medications": ["Sertraline 50mg daily", "Clonazepam 0.5mg"], "raw_text": "..." } } Extraction Schema Field Type Extraction Method patient_name string | null Heuristic label scan: Patient: / Name: date string | null Regex: DD/MM/YYYY · MM-DD-YY · DD-MM-YYYY vitals object Regex: BP · HR · Temp · SpO2 · BMI · Weight lab_values object Regex: HbA1c · Glucose · WBC · RBC · TSH · T3 · T4 medications array Lines containing dosage units: mg · mcg · ml diagnoses array Lines matching: diagnosis · impression · assessment   No file attached      File field present but empty

---

🔗 Combined Intelligence — Fusion Operations

Full-stack intelligence — emotion NLP + OCR extraction + medical cross-reference correlation in one call Field Value Method URL http://localhost:5000/api/analyze Content-Type multipart/form-data Returns Emotion analysis + OCR result + cross-reference matches curl -X POST http://localhost:5000/api/analyze \ -F "message=I feel exhausted and completely hopeless" \ -F "file=@/path/to/medical_report.png" 📤 Sample Response · 200 OK { "status": "success", "emotion_analysis": { "dominant_emotion": "sadness", "confidence": 84.2, "risk_level": "medium", "color": "blue", "advice": "Feelings of sadness are valid. Consider reaching out to a friend or therapist." }, "ocr_result": { "diagnoses": ["Impression: Major Depressive Disorder"], "medications": ["Sertraline 50mg daily"] }, "cross_reference": { "condition_matches": [ { "condition": "depression", "matched_symptoms": ["hopelessness", "fatigue", "low energy"], "known_medications": ["Sertraline", "Fluoxetine", "Bupropion"], "recommendation": "Consult a psychiatrist for a full clinical evaluation." } ], "emotion_health_correlation": "⚠️ Your emotional state aligns with detected medical indicators. Professional consultation is strongly advised.", "total_matches": 1 } } Cross-Reference Logic Trigger Condition Output emotion = sadness or fear + depression/anxiety in report ⚠️ Strong clinical correlation warning risk_level = high 🔴 Immediate professional referral message Medical keyword matches found 📋 Recommendations surfaced from DB No matches + low risk ✅ Continue monitoring your wellbeing   Neither message nor file provided

---

⚙️ System — Health & Monitoring

Service liveness probe — integrate with Docker HEALTHCHECK, Prometheus, or UptimeRobot for production monitoring Field Value Method URL http://localhost:5000/api/health Content-Type application/json Returns Service status confirmation curl -X GET http://localhost:5000/api/health 📤 Sample Response · 200 OK { "status": "ok", "service": "Mental Health AI API" } 💡 Use this endpoint in Docker HEALTHCHECK, GitHub Actions smoke tests, or UptimeRobot ping monitors to verify API availability in CI/CD and production deployments.

---

📦 Full Tech Stack

Layer	Tool / Library	Version	Role
🧠 Transformer Model	distilroberta-base (HuggingFace)	—	Real-time emotion NLP inference
🐍 Web Framework	Flask	2.x	Lightweight production REST API
🔗 CORS Middleware	flask-cors	latest	Cross-origin request handling
🖼️ OCR Engine	pytesseract	latest	Computer vision text extraction
🖼️ Image Processing	Pillow (PIL)	latest	Grayscale preprocessing for OCR accuracy
📡 ML Pipeline	transformers (HuggingFace)	4.x	NLP model loading + tokenisation
🌐 Frontend SPA	Vanilla HTML/CSS/JS	—	Dark-mode single-page application
🔒 Code Quality	flake8 + pre-commit	—	PEP8 enforcement + secret scanning

---

🪝 Pre-commit Hook Pipeline

🔍  detect private key ...................... ✅  Passed
🌿  don't commit to branch .................. ✅  Passed
🐍  flake8 .................................. ✅  Passed   ← E302 · E501 · E741 all resolved
📝  codespell ............................... ✅  Passed
🔐  mask secrets in staged files ............ ✅  Passed
⚙️   syntax and compile check ................ ✅  Passed

---

⚠️ Medical & Ethical Disclaimer

Caution

This project is built for educational, research, and portfolio demonstration purposes only. It is not a certified medical device and must not be used as a substitute for professional psychological or clinical advice, diagnosis, or treatment. If you or someone you know is experiencing a mental health crisis, please contact a qualified healthcare professional or a crisis helpline immediately.

---

🤝 Contributing

git checkout -b feature/your-feature-name
git commit -m "feat: describe your change clearly"
git push origin feature/your-feature-name
# Open a Pull Request on GitHub ↗

• ✅ Follow PEP8 — flake8 enforced via pre-commit
• ✅ Write descriptive commit messages (feat: · fix: · docs:)
• ✅ Add docstrings to all new functions
• ✅ Test all endpoints with curl or Postman before submitting PR

---

📄 Licensed under the MIT License