Skip to content

Commit f4f7132

Browse files
rauljuncoVrauljuncoV
committed
Add Git Second Brain: RAG app for repo commit history using Oracle AI Database 26ai
1 parent c5d10a2 commit f4f7132

14 files changed

Lines changed: 999 additions & 0 deletions

File tree

‎apps/git-second-brain/README.md‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
# Git Second Brain
2+
3+
A RAG (Retrieval-Augmented Generation) application that lets you ask
4+
natural-language questions about **any Git repository** by analysing its
5+
commit history. The included example uses the **FastAPI** open-source project.
6+
7+
Commits are embedded as vectors and stored in **Oracle AI Database 26ai**.
8+
At query time the most relevant commits are retrieved via `VECTOR_DISTANCE`
9+
and passed as context to an OpenAI model through **LangChain**, producing
10+
grounded answers with commit citations.
11+
12+
## Project structure
13+
14+
```
15+
git-second-brain/
16+
├── database/ # SQL scripts: user creation + schema setup
17+
├── data-loader/ # One-time ETL: parse commits, embed, load into Oracle 26ai
18+
├── app/ # Streamlit chat UI + LangChain RAG chain
19+
├── diffs/ # Pre-extracted per-commit diff files
20+
└── fastapi_commits.txt # Delimited commit metadata
21+
```
22+
23+
| Folder | Purpose | Details |
24+
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------- |
25+
| **database/** | SQL scripts to create the Oracle user, table, indexes, and (optionally) the vector index. | [database/README.md](database/README.md) |
26+
| **data-loader/** | Reads the extracted commit metadata and diff files, generates 384-dim vector embeddings with `sentence-transformers`, and bulk-inserts everything into Oracle 26ai. | [data-loader/README.md](data-loader/README.md) |
27+
| **app/** | Streamlit chat interface where users ask questions. A custom LangChain retriever queries Oracle 26ai vector search, and the retrieved commits are sent to OpenAI to generate a cited answer. | [app/README.md](app/README.md) |
28+
29+
## Extracting repo data
30+
31+
The examples below use **FastAPI**, but this works with **any Git repository**.
32+
33+
```bash
34+
# Clone the target repo
35+
git clone https://github.com/tiangolo/fastapi.git
36+
mkdir diffs
37+
cd fastapi
38+
39+
# Extract commit metadata with safe delimiters
40+
git log --all --no-merges \
41+
--pretty=format:"<<<COMMIT>>>%n%H%n%an%n%aI%n%s%n<<<BODY>>>%n%b%n<<<END>>>%n" \
42+
> ../fastapi_commits.txt
43+
44+
# Extract diff stats as a single file
45+
git log --all --no-merges \
46+
--pretty=format:"===SHA:%H===" --stat \
47+
> ../diffs/all_diffs.txt
48+
49+
cd ..
50+
```
51+
52+
> **Tip:** The data loader caps at 3 000 commits by default, which keeps
53+
> indexing time under 10 minutes and covers roughly 2015–today for FastAPI.
54+
55+
## Prerequisites
56+
57+
- Python 3.10+
58+
- Oracle AI Database 26ai (running and accessible)
59+
- OpenAI API key (for the chat app)
60+
61+
## Quick start
62+
63+
> **Important:** Load the environment variables from each folder's `.env` file
64+
> before running Python scripts. See each folder's README for details.
65+
66+
```bash
67+
# 0. Set up the database
68+
cd database
69+
sqlplus system/Welcome_123@//localhost:1521/FREEPDB1 @01_create_user.sql
70+
sqlplus system/Welcome_123@//localhost:1521/FREEPDB1 @02_create_schema.sql
71+
cd ..
72+
73+
# 1. Extract repo data (see "Extracting repo data" above)
74+
75+
# 2. Load data into Oracle 26ai
76+
cd data-loader
77+
python -m venv .venv && .venv\Scripts\activate # or source .venv/bin/activate
78+
pip install -r requirements.txt
79+
cp .env.example .env # fill in your Oracle credentials
80+
# load env vars, then:
81+
python load_data.py
82+
cd ..
83+
84+
# 3. Run the app
85+
cd app
86+
python -m venv .venv && .venv\Scripts\activate
87+
pip install -r requirements.txt
88+
cp .env.example .env # fill in Oracle + OpenAI credentials
89+
# load env vars, then:
90+
streamlit run app.py
91+
```
92+
93+
See each folder's README for full setup and configuration details.

‎apps/git-second-brain/app/.env.example‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Oracle AI Database 26ai connection
2+
ORACLE_USER=GITHUB_SECOND_BRAIN
3+
ORACLE_PASSWORD=<your-password>
4+
ORACLE_DSN=localhost:1521/FREEPDB1
5+
6+
# OpenAI (can also be entered in the Streamlit sidebar)
7+
OPENAI_API_KEY=sk-...

‎apps/git-second-brain/app/README.md‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Git Second Brain — App
2+
3+
Streamlit chat UI that lets you ask natural-language questions about a
4+
repository's commit history, powered by **Oracle AI Database 26ai Vector Search**,
5+
LangChain, and OpenAI.
6+
7+
## Architecture
8+
9+
```
10+
User question
11+
│
12+
▼
13+
┌────────────────────┐ ┌──────────────────────────┐
14+
│ Streamlit (app.py)│─────▶│ OracleCommitRetriever │
15+
│ Chat interface │ │ sentence-transformers │
16+
└────────┬───────────┘ │ + Oracle 26ai vector │
17+
│ │ VECTOR_DISTANCE search │
18+
│ context docs └──────────────────────────┘
19+
▼
20+
┌────────────────────┐
21+
│ LangChain RAG │
22+
│ ChatOpenAI (GPT) │
23+
└────────────────────┘
24+
```
25+
26+
## Prerequisites
27+
28+
| Requirement | Version |
29+
| ----------------------- | ----------------------------- |
30+
| Python | 3.10+ |
31+
| Oracle AI Database 26ai | Running and accessible |
32+
| OpenAI API key | Any `gpt-4o-mini` capable key |
33+
34+
The `data-loader/` must have been run first so the `FASTAPI_COMMITS` table is
35+
populated with embeddings.
36+
37+
## Setup
38+
39+
```bash
40+
cd app
41+
python -m venv .venv
42+
43+
# Windows
44+
.venv\Scripts\activate
45+
# Linux / macOS
46+
source .venv/bin/activate
47+
48+
pip install -r requirements.txt
49+
```
50+
51+
Copy `.env.example` to `.env` and fill in your credentials:
52+
53+
```bash
54+
cp .env.example .env
55+
```
56+
57+
## Running
58+
59+
The app reads Oracle credentials from environment variables. Load them before
60+
starting Streamlit:
61+
62+
```bash
63+
# Load env vars from .env (use your preferred method)
64+
# Windows PowerShell:
65+
Get-Content .env | ForEach-Object { if ($_ -match '^([^#].+?)=(.*)$') { [Environment]::SetEnvironmentVariable($Matches[1], $Matches[2]) } }
66+
67+
# Linux / macOS:
68+
# export $(grep -v '^#' .env | xargs)
69+
70+
streamlit run app.py
71+
```
72+
73+
The app opens at <http://localhost:8501>.
74+
75+
## Smoke test
76+
77+
A standalone script that verifies the vector-search round trip without
78+
Streamlit or OpenAI. Requires the same environment variables:
79+
80+
```bash
81+
python smoke_test.py
82+
```
83+
84+
## Files
85+
86+
| File | Purpose |
87+
| ------------------ | ------------------------------------------------------------- |
88+
| `app.py` | Streamlit chat UI + LangChain RAG chain |
89+
| `retriever.py` | LangChain `BaseRetriever` backed by Oracle 26ai vector search |
90+
| `smoke_test.py` | Minimal end-to-end connectivity & vector-search test |
91+
| `requirements.txt` | Pinned Python dependencies |
92+
| `.env.example` | Template for required environment variables |
93+
94+
## Environment variables
95+
96+
| Variable | Required | Default | Description |
97+
| ----------------- | -------- | ------- | ---------------------------------------------- |
98+
| `ORACLE_USER` | Yes | — | Database username |
99+
| `ORACLE_PASSWORD` | Yes | — | Database password |
100+
| `ORACLE_DSN` | Yes | — | Connect string, e.g. `localhost:1521/FREEPDB1` |
101+
| `OPENAI_API_KEY` | No | — | Can also be entered in the Streamlit sidebar |

‎apps/git-second-brain/app/app.py‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
"""
2+
Git Second Brain - Streamlit Chat UI
3+
Ask natural-language questions about FastAPI's commit history,
4+
powered by Oracle AI Database 26ai Vector Search + LangChain + OpenAI.
5+
6+
Run:
7+
streamlit run app.py
8+
"""
9+
10+
import os
11+
12+
import streamlit as st
13+
from langchain_core.output_parsers import StrOutputParser
14+
from langchain_core.prompts import ChatPromptTemplate
15+
from langchain_openai import ChatOpenAI
16+
from retriever import OracleCommitRetriever
17+
18+
# ========================= Page config =========================
19+
st.set_page_config(
20+
page_title="Git Second Brain",
21+
page_icon="🧠",
22+
layout="wide",
23+
)
24+
25+
# ========================= Sidebar =============================
26+
with st.sidebar:
27+
st.title("Git Second Brain")
28+
st.caption("Oracle AI Database 26ai + LangChain + OpenAI")
29+
30+
openai_key = st.text_input(
31+
"OpenAI API Key",
32+
type="password",
33+
value=os.getenv("OPENAI_API_KEY", ""),
34+
help="Stored only in this session, never persisted.",
35+
)
36+
37+
model_name = st.selectbox(
38+
"Model",
39+
["gpt-4o-mini", "gpt-4o", "gpt-4.1-mini", "gpt-4.1-nano"],
40+
index=0,
41+
)
42+
43+
top_k = st.slider("Commits to retrieve", min_value=3, max_value=15, value=8)
44+
45+
temperature = st.slider("Temperature", min_value=0.0, max_value=1.0, value=0.2, step=0.05)
46+
47+
st.divider()
48+
st.markdown(
49+
"**How it works**\n\n"
50+
"1. Your question is embedded with sentence-transformers\n"
51+
"2. Oracle 26ai runs `VECTOR_DISTANCE` to find the most relevant commits\n"
52+
"3. LangChain passes those commits as context to OpenAI\n"
53+
"4. You get a grounded answer with commit citations"
54+
)
55+
56+
st.divider()
57+
st.markdown("**Sample questions**")
58+
sample_questions = [
59+
"Why did FastAPI switch to Pydantic v2?",
60+
"How has dependency injection evolved?",
61+
"What were the biggest breaking changes in the last 2 years?",
62+
"When did lifespan replace startup/shutdown events?",
63+
"What security fixes were applied recently?",
64+
]
65+
for q in sample_questions:
66+
if st.button(q, use_container_width=True):
67+
st.session_state["prefill"] = q
68+
69+
# ========================= System prompt =======================
70+
SYSTEM_PROMPT = """\
71+
You are Git Second Brain, an AI assistant that answers questions about the
72+
FastAPI open-source project by analyzing its Git commit history.
73+
74+
You will receive a set of relevant commits retrieved from Oracle AI Database 26ai
75+
via vector similarity search. Use ONLY these commits to answer the question.
76+
If the commits do not contain enough information, say so honestly.
77+
78+
Rules:
79+
- Cite specific commits by their short SHA and date when supporting a claim.
80+
- Summarize the narrative arc when multiple commits tell a story.
81+
- Keep answers concise but thorough (3-6 paragraphs max).
82+
- If you are unsure, say "Based on the commits I found..." to hedge.
83+
- Never invent commit SHAs or dates.
84+
"""
85+
86+
RAG_TEMPLATE = ChatPromptTemplate.from_messages(
87+
[
88+
("system", SYSTEM_PROMPT),
89+
("human", "Retrieved commits:\n\n{context}\n\n---\nQuestion: {question}"),
90+
]
91+
)
92+
93+
# ========================= Init state ==========================
94+
if "messages" not in st.session_state:
95+
st.session_state.messages = []
96+
97+
if "retriever" not in st.session_state:
98+
with st.spinner("Connecting to Oracle AI Database 26ai ..."):
99+
st.session_state.retriever = OracleCommitRetriever(top_k=top_k)
100+
101+
# ========================= Chat display ========================
102+
st.header("Ask your repo anything")
103+
104+
for msg in st.session_state.messages:
105+
with st.chat_message(msg["role"]):
106+
st.markdown(msg["content"])
107+
if msg.get("sources"):
108+
with st.expander(f"Retrieved commits ({len(msg['sources'])})"):
109+
for doc in msg["sources"]:
110+
meta = doc.metadata
111+
st.markdown(
112+
f"**`{meta['sha'][:10]}`** | {meta['date']} | "
113+
f"*{meta['author']}*\n\n"
114+
f"> {meta['subject']}"
115+
)
116+
st.divider()
117+
118+
# ========================= Chat input ==========================
119+
prefill = st.session_state.pop("prefill", None)
120+
user_input = st.chat_input("Ask about FastAPI's history ...") or prefill
121+
122+
if user_input:
123+
if not openai_key:
124+
st.error("Please enter your OpenAI API key in the sidebar.")
125+
st.stop()
126+
127+
# Show user message
128+
st.session_state.messages.append({"role": "user", "content": user_input})
129+
with st.chat_message("user"):
130+
st.markdown(user_input)
131+
132+
# Retrieve from Oracle 26ai
133+
with st.chat_message("assistant"):
134+
with st.spinner("Searching Oracle 26ai Vector Search ..."):
135+
retriever = st.session_state.retriever
136+
retriever.top_k = top_k
137+
docs = retriever.invoke(user_input)
138+
139+
context = "\n\n---\n\n".join(doc.page_content for doc in docs)
140+
141+
# LangChain RAG chain
142+
llm = ChatOpenAI(
143+
model=model_name,
144+
temperature=temperature,
145+
api_key=openai_key,
146+
)
147+
chain = RAG_TEMPLATE | llm | StrOutputParser()
148+
149+
with st.spinner("Generating answer ..."):
150+
answer = chain.invoke({"context": context, "question": user_input})
151+
152+
st.markdown(answer)
153+
154+
# Show retrieved commits
155+
with st.expander(f"Retrieved commits ({len(docs)})"):
156+
for doc in docs:
157+
meta = doc.metadata
158+
st.markdown(
159+
f"**`{meta['sha'][:10]}`** | {meta['date']} | "
160+
f"*{meta['author']}*\n\n"
161+
f"> {meta['subject']}"
162+
)
163+
st.divider()
164+
165+
st.session_state.messages.append(
166+
{
167+
"role": "assistant",
168+
"content": answer,
169+
"sources": docs,
170+
}
171+
)

‎apps/git-second-brain/app/requirements.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
oracledb>=2.2.0,<4
2+
sentence-transformers>=5.0,<6
3+
langchain>=1.2,<2
4+
langchain-core>=1.2,<2
5+
langchain-openai>=1.1,<2
6+
streamlit>=1.38,<2

0 commit comments

Comments
 (0)