VOnn Style Intelligence - AI Fashion Stylist & Virtual Try-On

VOnn is an end-to-end Generative AI Fashion System that acts as a professional personal stylist. It analyzes a user's physical profile from a photo, understands their specific occasion context (e.g., "minimalist linen for a beach wedding"), and reasons through multiple layers of style logic to recommend the perfect outfit. Finally, it uses Virtual Try-On (VTO) technology to visualize those recommendations on the user's actual body.

Live Demo: https://thesigma.fit

---

🚀 The 5-Layer Cognitive Architecture

VOnn moves beyond simple image similarity. It uses a chain-of-thought reasoning pipeline designed to mimic a human stylist's decision-making process.

1. Layer 1: Input Analysis (Physical & Contextual)

   • Biometrics: Uses Gemini 1.5 Flash (Vision) to extract gender, age group, body shape (7-type system), visual weight, and skin undertone from the user's photo.
   • Context: Extracts the canonical occasion (e.g., "wedding", "interview") and environmental constraints (weather, indoor/outdoor) from the user's text query.

2. Layer 2: Style Reasoning (User $\to$ Style)

   • Logic: Translates physical attributes into abstract style rules.
   • Example: "For a trapezoid body shape at a formal event, prioritize high shoulder structure but medium visual loudness."
   • Expert Advisory: Provides a reasoning summary, explaining why certain choices were made (e.g., "Muted colors selected to complement your cool undertone").

3. Layer 3: Fashion Translation (Style $\to$ Fashion)

   • Logic: Converts abstract style rules into concrete garment attributes.
   • Example: "High Structure + Formal $\to$ Tuxedo / Blazer."
   • Context Persistence: Explicit user requests (e.g., "I want pink") are prioritized here as hard constraints, overriding default style rules if necessary.

4. Layer 4: Catalog Retrieval (Semantic Search)

   • Hybrid Search: Embeds both the Layer 3 attributes and the raw user intent into a single vector.
   • Hard Constraints: Enforces strict filtering (e.g., WHERE gender_target = 'male') to ensure relevance.
   • Engine: Powered by BigQuery Vector Search using text-embedding-004.

5. Layer 5: Virtual Try-On (VTO)

   • Generation: Uses Google's Vertex AI Virtual Try-On model to generate a high-fidelity image of the user wearing the recommended product.
   • Preservation: Maintains user identity, pose, and body shape while realistically draping the garment.

---

🛠 Technology Stack

• Core AI: Google Vertex AI (Gemini 1.5 Flash, Imagen 3, Virtual Try-On API).
• Embeddings: text-embedding-004.
• Database & Vector Store: Google BigQuery (for catalog, sessions, and vector index).
• Backend Logic: Python (Pydantic for strict data validation).
• Frontend: Streamlit.
• Infrastructure: Google Cloud Run (Serverless), Artifact Registry, Cloud Build.
• Storage: Google Cloud Storage (for images).

---

📂 Project Structure

.
├── app/
│   ├── fashion/          # Layer 3: Style -> Fashion translation
│   ├── models/           # Pydantic data models (validation layer)
│   ├── products/         # Layer 4: Catalog retrieval logic
│   ├── styling/          # Layer 2: User -> Style translation
│   ├── user/             # Layer 1: Biometric analysis
│   ├── vto/              # Layer 5: Virtual Try-On API integration
│   ├── pipeline_engine.py # The core orchestrator class
│   ├── tracking.py       # BigQuery logging & analytics
│   ├── init_*.py         # DB initialization scripts
│   └── merge_tables.py   # Analytics aggregation script
├── embedding/            # Offline batch jobs (Catalog enrichment)
├── credentials/          # Local service account keys (gitignored)
├── interface.py          # Streamlit UI entry point
├── main.py               # CLI entry point (for testing)
├── config.py             # Environment & Project config
├── deploy.sh             # Cloud Run deployment script
├── Dockerfile            # Container definition
└── requirements.txt      # Python dependencies

---

⚡ Setup & Installation

1. Prerequisites

• Python 3.10+
• Google Cloud Project with Billing Enabled.
• APIs Enabled: Vertex AI, BigQuery, Cloud Run, Artifact Registry, Cloud Build.

2. Environment Variables

Create a .env file or export these variables:

export GOOGLE_CLOUD_PROJECT="vonn-476213"
export OPENWEATHER_API_KEY="your_api_key"
export GOOGLE_APPLICATION_CREDENTIALS="path/to/your/sa-key.json" # For local dev

3. Installation

git clone https://github.com/jaychitransh007/TheSigmaFit4.0.0.git
cd TheSigmaFit4.0.0
pip install -r requirements.txt

---

🏃 Running Locally

Web UI (Streamlit)

This is the main interface for the application.

streamlit run interface.py

Access at http://localhost:8501

CLI Pipeline (Test/Debug)

Run the pipeline without the UI to test logic layers.

python main.py test/user/mj.png "bright pink linen outfit for a beach party" --style-examples True --fashion-examples False --run-until catalog

---

☁️ Deployment (Google Cloud Run)

The project includes a deploy.sh script that handles containerization and deployment to Cloud Run.

1. Configure Deployment: Edit deploy.sh to set your specific PROJECT_ID and REGION.

2. Deploy:

   ./deploy.sh

   This will build the Docker image, push it to Artifact Registry, and deploy the service to Cloud Run.

3. Domain Mapping:

   • Go to Cloud Run Console -> Manage Custom Domains.
   • Map thesigma.fit to your service.
   • Update DNS records (A/AAAA) at your registrar (e.g., GoDaddy, Cloudflare).

---

📊 Data & Tracking (BigQuery)

The system maintains a comprehensive audit trail in BigQuery dataset: style_intelligence_golden_data.

Tables:

1. user_sessions: High-level log of every run (Inputs, AI logic, summaries).
   • Purpose: Debugging AI reasoning ("Why did it suggest this?").
2. user_interactions: Granular log of every product matched and VTO generated.
   • Key: interaction_id (Unique ID for every card shown to user).
3. user_feedback: Logs user thumbs up/down actions.
   • Key: interaction_id (Foreign key linking back to the specific product interaction).
4. catalog_products: The master inventory with Vector Embeddings.

Analytics

You can join these tables to analyze performance:

-- Join Feedback with Product Details
SELECT 
    i.product_id, 
    f.rating, 
    i.input_context 
FROM `user_interactions` i
JOIN `user_feedback` f 
ON i.interaction_id = f.interaction_id
WHERE f.rating = 'down'

---

📝 License

Proprietary - VOnn Style Intelligence.