Skip to content

Commit 017281b

Browse files
fix: Correct formatting in README.md for improved clarity and consistency
1 parent f92c197 commit 017281b

1 file changed

Lines changed: 52 additions & 44 deletions

File tree

README.md

Lines changed: 52 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ A terminal chat client for running **local LLMs on consumer hardware**. Chat wit
1515

1616
Perfect for developers who want a **self-hosted ChatGPT alternative** running on their gaming PC or homelab server. Also useful for local AI coding assistants, agentic workflows, and agent development.
1717

18-
> Named after ZORAC, the intelligent Ganymean computer from James P. Hogan's *The Gentle Giants of Ganymede*.
18+
> Named after ZORAC, the intelligent Ganymean computer from James P. Hogan's _The Gentle Giants of Ganymede_.
1919
2020
## Install
2121

@@ -68,7 +68,7 @@ That's it. Your conversation is automatically saved and restored between session
6868

6969
### Rich Terminal UI with Live Streaming
7070

71-
*Interactive chat with real-time streaming responses, markdown rendering, and performance metrics*
71+
_Interactive chat with real-time streaming responses, markdown rendering, and performance metrics_
7272

7373
![Zorac Chat Interface](https://raw.githubusercontent.com/chris-colinsky/Zorac/main/screenshots/zorac-screenshot-1.png)
7474

@@ -90,59 +90,59 @@ That's it. Your conversation is automatically saved and restored between session
9090

9191
All commands start with `/` and auto-complete with Tab:
9292

93-
| Command | Description |
94-
|---------|-------------|
95-
| `/help` | Show all available commands |
96-
| `/clear` | Reset conversation and start fresh |
97-
| `/save` | Manually save the current session |
98-
| `/load` | Reload session from disk (discards unsaved changes) |
99-
| `/tokens` | Show current token usage, limits, and remaining capacity |
100-
| `/summarize` | Force summarization of conversation history |
101-
| `/summary` | Display the current conversation summary |
102-
| `/reconnect` | Retry connection to the vLLM server |
103-
| `/config list` | Show all current settings |
104-
| `/config set KEY VALUE` | Update a setting (takes effect immediately) |
105-
| `/config get KEY` | Show a specific setting value |
106-
| `/quit` or `/exit` | Save session and exit |
107-
| `Ctrl+C` | Interrupt a streaming response without exiting |
93+
| Command | Description |
94+
| ----------------------- | -------------------------------------------------------- |
95+
| `/help` | Show all available commands |
96+
| `/clear` | Reset conversation and start fresh |
97+
| `/save` | Manually save the current session |
98+
| `/load` | Reload session from disk (discards unsaved changes) |
99+
| `/tokens` | Show current token usage, limits, and remaining capacity |
100+
| `/summarize` | Force summarization of conversation history |
101+
| `/summary` | Display the current conversation summary |
102+
| `/reconnect` | Retry connection to the vLLM server |
103+
| `/config list` | Show all current settings |
104+
| `/config set KEY VALUE` | Update a setting (takes effect immediately) |
105+
| `/config get KEY` | Show a specific setting value |
106+
| `/quit` or `/exit` | Save session and exit |
107+
| `Ctrl+C` | Interrupt a streaming response without exiting |
108108

109109
You can also ask the assistant about commands in natural language — the LLM is aware of all Zorac functionality.
110110

111111
## Configuration
112112

113113
All settings can be changed at runtime without restarting:
114114

115-
```
115+
```text
116116
You: /config set TEMPERATURE 0.7
117117
✓ Updated TEMPERATURE in ~/.zorac/config.json
118118
✓ Temperature will take effect on next message.
119119
```
120120

121121
### All Settings
122122

123-
| Setting | Default | Description |
124-
|---------|---------|-------------|
125-
| **Server** | | |
126-
| `VLLM_BASE_URL` | `http://localhost:8000/v1` | vLLM server endpoint |
127-
| `VLLM_API_KEY` | `EMPTY` | API key (vLLM doesn't require one) |
128-
| `VLLM_MODEL` | `dark-side-of-the-code/Mistral-Small-24B-Instruct-2501-AWQ` | Model to use |
129-
| **Model Parameters** | | |
130-
| `TEMPERATURE` | `0.1` | Randomness: `0.0` = deterministic, `0.7` = balanced, `1.0`+ = creative |
131-
| `MAX_OUTPUT_TOKENS` | `4000` | Maximum tokens per response |
132-
| `STREAM` | `true` | Real-time streaming (`true`) or wait for complete response (`false`) |
133-
| **Context Management** | | |
134-
| `MAX_INPUT_TOKENS` | `12000` | Token budget for system prompt + conversation history |
135-
| `KEEP_RECENT_MESSAGES` | `6` | Messages preserved when auto-summarization triggers |
136-
| **Display** | | |
137-
| `CODE_THEME` | `monokai` | Pygments syntax highlighting theme for code blocks |
138-
| **Advanced** | | |
139-
| `TIKTOKEN_ENCODING` | `cl100k_base` | Token counting encoding (match to your model family) |
123+
| Setting | Default | Description |
124+
| ---------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------- |
125+
| **Server** | | |
126+
| `VLLM_BASE_URL` | `http://localhost:8000/v1` | vLLM server endpoint |
127+
| `VLLM_API_KEY` | `EMPTY` | API key (vLLM doesn't require one) |
128+
| `VLLM_MODEL` | `dark-side-of-the-code/Mistral-Small-24B-Instruct-2501-AWQ` | Model to use |
129+
| **Model Parameters** | | |
130+
| `TEMPERATURE` | `0.1` | Randomness: `0.0` = deterministic, `0.7` = balanced, `1.0`+ = creative |
131+
| `MAX_OUTPUT_TOKENS` | `4000` | Maximum tokens per response |
132+
| `STREAM` | `true` | Real-time streaming (`true`) or wait for complete response (`false`) |
133+
| **Context Management** | | |
134+
| `MAX_INPUT_TOKENS` | `12000` | Token budget for system prompt + conversation history |
135+
| `KEEP_RECENT_MESSAGES` | `6` | Messages preserved when auto-summarization triggers |
136+
| **Display** | | |
137+
| `CODE_THEME` | `monokai` | Pygments syntax highlighting theme for code blocks |
138+
| **Advanced** | | |
139+
| `TIKTOKEN_ENCODING` | `cl100k_base` | Token counting encoding (match to your model family) |
140140

141141
**Popular code themes:** `monokai`, `dracula`, `github-dark`, `one-dark`, `solarized-dark`, `solarized-light`, `nord`, `gruvbox-dark`, `native`
142142

143143
### View Current Configuration
144144

145-
```
145+
```text
146146
You: /config list
147147
148148
Configuration:
@@ -180,7 +180,7 @@ Source users can also use a `.env` file in the project root. See [Configuration
180180

181181
Zorac tracks tokens to stay within your model's context window:
182182

183-
```
183+
```text
184184
You: /tokens
185185
📊 Token usage:
186186
Current: ~3421 tokens
@@ -204,14 +204,14 @@ When the conversation exceeds `MAX_INPUT_TOKENS`, Zorac automatically summarizes
204204

205205
Runs on **consumer gaming GPUs**:
206206

207-
| GPU | VRAM | Model Size | Performance |
208-
|-----|------|------------|-------------|
209-
| RTX 4090 | 24GB | Up to 24B (AWQ) | 60-65 tok/s |
207+
| GPU | VRAM | Model Size | Performance |
208+
| ----------- | ---- | --------------- | ----------- |
209+
| RTX 4090 | 24GB | Up to 24B (AWQ) | 60-65 tok/s |
210210
| RTX 3090 Ti | 24GB | Up to 24B (AWQ) | 55-60 tok/s |
211-
| RTX 3090 | 24GB | Up to 24B (AWQ) | 55-60 tok/s |
212-
| RTX 4080 | 16GB | Up to 14B (AWQ) | 45-50 tok/s |
213-
| RTX 4070 Ti | 12GB | Up to 7B (AWQ) | 40-45 tok/s |
214-
| RTX 3080 | 10GB | Up to 7B (AWQ) | 35-40 tok/s |
211+
| RTX 3090 | 24GB | Up to 24B (AWQ) | 55-60 tok/s |
212+
| RTX 4080 | 16GB | Up to 14B (AWQ) | 45-50 tok/s |
213+
| RTX 4070 Ti | 12GB | Up to 7B (AWQ) | 40-45 tok/s |
214+
| RTX 3080 | 10GB | Up to 7B (AWQ) | 35-40 tok/s |
215215

216216
See [Server Setup](https://zorac.lowgravitylab.com/SERVER_SETUP/) for optimization details.
217217

@@ -240,48 +240,56 @@ You can use any vLLM-compatible model (Llama, Qwen, Phi, DeepSeek, etc.) by chan
240240
<summary><b>Can I run this without a GPU?</b></summary>
241241

242242
No, this requires an NVIDIA GPU with at least 10GB VRAM. CPU-only inference is too slow for interactive chat.
243+
243244
</details>
244245

245246
<details>
246247
<summary><b>How does this compare to Ollama?</b></summary>
247248

248249
Zorac uses vLLM for faster inference (60+ tok/s vs Ollama's 20-30 tok/s on the same hardware) and supports advanced features like tool calling for agentic workflows. Ollama is easier to set up but slower.
250+
249251
</details>
250252

251253
<details>
252254
<summary><b>Do I need to be online?</b></summary>
253255

254256
Only for the initial model download (~14GB for Mistral-24B-AWQ). After that, everything runs completely offline.
257+
255258
</details>
256259

257260
<details>
258261
<summary><b>Is this legal? Can I use this commercially?</b></summary>
259262

260263
Yes. Mistral-Small is Apache 2.0 licensed (free commercial use). vLLM is also Apache 2.0.
264+
261265
</details>
262266

263267
<details>
264268
<summary><b>What about AMD GPUs or Mac M-series?</b></summary>
265269

266270
This is specifically for NVIDIA GPUs using CUDA. For AMD, you'd need ROCm support (experimental). For Mac M-series, check out MLX or llama.cpp instead.
271+
267272
</details>
268273

269274
<details>
270275
<summary><b>How much does it cost to run?</b></summary>
271276

272277
Electricity for an RTX 4090 at ~300W is roughly $0.05-0.10 per hour. Far cheaper than API costs for heavy usage.
278+
273279
</details>
274280

275281
<details>
276282
<summary><b>How do I copy text from the chat?</b></summary>
277283

278284
Zorac uses mouse reporting for scrolling, which can interfere with native text selection in some terminals. In **iTerm2**, hold **Option (⌥)** while clicking and dragging to select text, then copy with **Cmd+C** as usual. Most terminals support a similar modifier key — check your terminal's documentation for its equivalent.
285+
279286
</details>
280287

281288
<details>
282289
<summary><b>What other models can I run?</b></summary>
283290

284291
Any model with vLLM support: Llama, Qwen, Phi, DeepSeek, etc. Just change the `VLLM_MODEL` setting. See [vLLM supported models](https://docs.vllm.ai/en/stable/models/supported_models/).
292+
285293
</details>
286294

287295
## Documentation

0 commit comments

Comments
 (0)