|
| 1 | +# OpenTela |
| 2 | + |
| 3 | +A decentralized distributed computing platform that orchestrates GPU resources across a peer-to-peer network for LLM serving. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +OpenTela connects GPU resources into a unified pool using: |
| 8 | +- **libp2p networking** for peer-to-peer communication |
| 9 | +- **CRDT-based state management** for distributed consensus |
| 10 | +- **Identity group routing** for intelligent request distribution |
| 11 | +- **Solana settlement** for automated usage billing |
| 12 | + |
| 13 | +Primary use case: Distributed GPU node orchestration for LLM serving, powering projects like the [SwissAI Initiative](https://serving.swissai.cscs.ch/). |
| 14 | + |
| 15 | +## Quick Start |
| 16 | + |
| 17 | +### Installation |
| 18 | + |
| 19 | +```bash |
| 20 | +# x86_64 |
| 21 | +wget https://github.com/eth-easl/OpenTela/releases/latest/download/otela-amd64 -O otela && chmod +x otela |
| 22 | + |
| 23 | +# arm64 |
| 24 | +wget https://github.com/eth-easl/OpenTela/releases/latest/download/otela-arm64 -O otela && chmod +x otela |
| 25 | +``` |
| 26 | + |
| 27 | +### Spin Up a Cluster |
| 28 | + |
| 29 | +**Head Node:** |
| 30 | +```bash |
| 31 | +./otela start --mode standalone --public-addr {YOUR_IP} --seed 0 |
| 32 | +``` |
| 33 | + |
| 34 | +**Worker Node:** |
| 35 | +```bash |
| 36 | +./otela start \ |
| 37 | + --bootstrap.addr /ip4/{HEAD_IP}/tcp/43905/p2p/{HEAD_PEER_ID} \ |
| 38 | + --subprocess "vllm serve Qwen/Qwen3-8B --port 8080" \ |
| 39 | + --service.name llm \ |
| 40 | + --service.port 8080 |
| 41 | +``` |
| 42 | + |
| 43 | +### Send Requests |
| 44 | + |
| 45 | +```python |
| 46 | +import openai |
| 47 | +client = openai.OpenAI( |
| 48 | + base_url="http://{HEAD_IP}:8092/v1/service/llm/v1", |
| 49 | + api_key="test-token" |
| 50 | +) |
| 51 | +response = client.chat.completions.create( |
| 52 | + model="Qwen/Qwen3-8B", |
| 53 | + messages=[{"role": "user", "content": "Hello!"}] |
| 54 | +) |
| 55 | +``` |
| 56 | + |
| 57 | +## Documentation |
| 58 | + |
| 59 | +Full documentation is available at the [OpenTela Docs](https://docs.opentela.ai) or browse the [`content/docs/`](content/docs/) directory: |
| 60 | + |
| 61 | +- **Tutorial** — Installation, spinup, routing, wallets, settlement |
| 62 | +- **Advanced** — CRDT internals, performance, security |
| 63 | +- **Blog** — Real-world deployments (SwissAI) |
| 64 | +- **Proposals** — Design documents |
| 65 | + |
| 66 | +## Development |
| 67 | + |
| 68 | +This repository contains the documentation site built with Next.js and Fumadocs. |
| 69 | + |
| 70 | +```bash |
| 71 | +npm install |
| 72 | +npm run dev |
| 73 | +``` |
| 74 | + |
| 75 | +The OpenTela binary source code is available at [eth-easl/OpenTela](https://github.com/eth-easl/OpenTela). |
| 76 | + |
| 77 | +## License |
| 78 | + |
| 79 | +MIT |
0 commit comments