Skip to content

Commit a5546a6

Browse files
pingsutwCopilot
pingsutw
and
Copilot
authored
docs: add devbox-based local development guide (#7294)
* docs: add devbox-based local development guide Document how to run a full Flyte stack locally using `make devbox-build`, `make devbox-run FLYTE_DEV=true`, and `make -C manager run`. Update manager/README.md to reflect the current service set (no Queue/State service), the PostgreSQL backend, and the actual API surface, and add a matching "Running Flyte Locally" section to CONTRIBUTING.md. Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * Update manager/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com> * Update manager/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com> * Update CONTRIBUTING.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@gmail.com> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Kevin Su <pingsutw@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 6481731 commit a5546a6

3 files changed

Lines changed: 115 additions & 182 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 83 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,10 @@ Before contributing, ensure you have:
1919
- [Buf CLI](https://buf.build/docs/installation) installed
2020
- Go 1.24.6 or later
2121
- Node.js and npm (for TypeScript)
22-
- Python 3.9+ with `uv` package manager
22+
- Python 3.10+ with `uv` package manager
2323
- Rust toolchain (if working with Rust bindings)
2424
- Git configured with your name and email
25+
- Docker (for building and running the devbox image)
2526

2627
### Setting Up Your Environment
2728

@@ -44,6 +45,87 @@ Before contributing, ensure you have:
4445
make gen
4546
```
4647

48+
## Running Flyte Locally
49+
50+
The fastest way to run a full Flyte stack on your machine is the bundled **devbox** — a k3d-based Kubernetes cluster with all dependencies (TaskAction CRD, Knative, PostgreSQL, etc.) pre-installed — combined with a locally-running `flyte-manager` binary.
51+
52+
### Start the Flyte Devbox
53+
54+
From the repo root:
55+
56+
```bash
57+
# Build the devbox image (first time only, or after Dockerfile changes)
58+
make devbox-build
59+
60+
# Start the devbox cluster in dev mode (required for running the manager locally)
61+
make devbox-run FLYTE_DEV=true
62+
63+
# Stop the devbox when you're done
64+
make devbox-stop
65+
```
66+
67+
`FLYTE_DEV=true` is required when you intend to run the manager locally — it disables the in-cluster manager so your local process can take over. `make devbox-run` writes a kubeconfig pointing at the devbox cluster in global kubeconfig, so `kubectl` will target it automatically.
68+
69+
### Build and Run the Manager
70+
71+
With the devbox running, start the manager locally:
72+
73+
```bash
74+
# From the repo root
75+
make -C manager run
76+
77+
# Or from manager/
78+
make run
79+
80+
# Or build and run the binary directly
81+
cd manager
82+
make build
83+
./bin/flyte-manager --config config.yaml
84+
```
85+
86+
The manager will:
87+
1. Connect to PostgreSQL and run database migrations
88+
2. Start all services in parallel goroutines
89+
3. Connect to your Kubernetes cluster
90+
4. Begin reconciling TaskAction CRs
91+
92+
### Configuration
93+
94+
Edit `manager/config.yaml`:
95+
96+
```yaml
97+
manager:
98+
# Single server port hosting all Connect services (Runs, Actions, DataProxy, Events, Cache, Secret, App).
99+
server:
100+
host: "0.0.0.0"
101+
port: 8090
102+
103+
executor:
104+
healthProbePort: 8081
105+
106+
kubernetes:
107+
namespace: "flyte"
108+
# Optional: specify custom kubeconfig path
109+
# kubeconfig: "/path/to/kubeconfig"
110+
111+
runs:
112+
storagePrefix: "s3://flyte-data"
113+
database:
114+
postgres:
115+
host: "localhost"
116+
port: 30001
117+
dbname: "runs"
118+
username: "postgres"
119+
password: "postgres"
120+
options: "sslmode=disable"
121+
122+
logger:
123+
level: 4 # Info level
124+
show-source: true
125+
```
126+
127+
See [`manager/README.md`](manager/README.md) for the full architecture, API endpoints, and troubleshooting tips.
128+
47129
## Development Workflow
48130

49131
### Creating a Feature Branch

‎manager/README.md‎

Lines changed: 31 additions & 180 deletions
Original file line numberDiff line numberDiff line change
@@ -2,132 +2,39 @@
22

33
The Flyte Manager is a unified binary that runs all Flyte services in a single process:
44

5-
- **Runs Service** (port 8090) - Manages workflow runs and action state
6-
- **Queue Service** (port 8089) - Creates and manages TaskAction CRs in Kubernetes
7-
- **Executor/Operator** (port 8081 health) - Reconciles TaskAction CRs and transitions them through states
5+
- **Runs Service** - Manages workflow runs and action state
6+
- **Executor/Operator** - Reconciles and transitions TaskAction CRs through their lifecycle
7+
- **Actions Service** - Serves action metadata and lifecycle APIs, including enqueueing TaskAction CRs
8+
- **DataProxy Service** - Proxies signed-URL and blob access for task I/O
9+
- **Events Service** - Ingests and fans out task/run events
10+
- **Cache Service** - Backs task output caching and lookups
11+
- **App Service** (+ internal proxy) - Hosts the Flyte UI/app and routes to internal services
12+
- **Secret Service** - Manages secret references used by tasks
813

914
## Features
1015

1116
✅ **Single Binary** - One process to deploy and manage
12-
✅ **Single SQLite Database** - All data in one file
17+
✅ **PostgreSQL Backend** - Shared database for all services
1318
✅ **Auto Kubernetes Detection** - Uses current kubeconfig
1419
✅ **Unified Configuration** - One config file for all services
1520
✅ **HTTP/2 Support** - Buf Connect compatible
1621

17-
## Quick Start
18-
19-
### Prerequisites
20-
21-
1. **Kubernetes cluster** (k3d, kind, minikube, or any cluster)
22-
2. **Go 1.24 or later**
23-
3. **TaskAction CRD** installed in the cluster
24-
4. **Kubeconfig** configured (or running in-cluster)
25-
26-
### Install TaskAction CRD
27-
28-
```bash
29-
kubectl apply -f ../executor/config/crd/bases/flyte.org_taskactions.yaml
30-
```
31-
32-
### Build and Run
33-
34-
```bash
35-
# Build the binary
36-
make build
37-
38-
# Run the manager
39-
make run
40-
41-
# Or run directly
42-
./bin/flyte-manager --config config.yaml
43-
```
44-
45-
The manager will:
46-
1. Initialize a SQLite database (`flyte.db`)
47-
2. Run database migrations
48-
3. Start all three services in parallel goroutines
49-
4. Connect to your Kubernetes cluster
50-
5. Begin reconciling TaskAction CRs
51-
52-
## Configuration
53-
54-
Edit `config.yaml`:
55-
56-
```yaml
57-
manager:
58-
runsService:
59-
host: "0.0.0.0"
60-
port: 8090
61-
62-
queueService:
63-
host: "0.0.0.0"
64-
port: 8089
65-
66-
executor:
67-
healthProbePort: 8081
68-
69-
kubernetes:
70-
namespace: "flyte"
71-
# Optional: specify custom kubeconfig path
72-
# kubeconfig: "/path/to/kubeconfig"
73-
74-
database:
75-
type: "sqlite"
76-
sqlite:
77-
file: "flyte.db"
78-
79-
logger:
80-
level: 4 # Info level
81-
show-source: true
82-
```
83-
84-
## Architecture
85-
86-
```
87-
┌─────────────────────────────────────────────────────────┐
88-
│ Flyte Manager Process │
89-
├─────────────────────────────────────────────────────────┤
90-
│ │
91-
│ ┌──────────────┐ ┌───────────────┐ ┌──────────────┐ │
92-
│ │ Runs Service │ │ Queue Service │ │ Executor │ │
93-
│ │ :8090 │ │ :8089 │ │ │ │
94-
│ │ │ │ │ │ Reconciles │ │
95-
│ │ - RunService │ │ Creates K8s │ │ TaskActions │ │
96-
│ │ - StateServ. │ │ TaskAction CRs│ │ │ │
97-
│ └──────┬───────┘ └───────┬───────┘ └──────┬───────┘ │
98-
│ │ │ │ │
99-
│ └──────────────────┴──────────────────┘ │
100-
│ │ │
101-
│ ┌────────┴────────┐ │
102-
│ │ SQLite DB │ │
103-
│ │ flyte.db │ │
104-
│ └─────────────────┘ │
105-
└─────────────────────────────────────────────────────────┘
106-
│
107-
↓
108-
Kubernetes Cluster
109-
(TaskAction CRs)
110-
```
111-
11222
## API Endpoints
11323

114-
### Runs Service (port 8090)
24+
### Manager (port 8090)
11525

116-
- `POST /flyteidl2.workflow.RunService/CreateRun` - Create a new run
117-
- `POST /flyteidl2.workflow.RunService/GetRun` - Get run details
118-
- `POST /flyteidl2.workflow.RunService/ListRuns` - List runs
119-
- `POST /flyteidl2.workflow.RunService/AbortRun` - Abort a run
120-
- `POST /flyteidl2.workflow.StateService/Put` - Update action state
121-
- `POST /flyteidl2.workflow.StateService/Get` - Get action state
122-
- `POST /flyteidl2.workflow.StateService/Watch` - Watch state updates
123-
- `GET /healthz` - Health check
124-
- `GET /readyz` - Readiness check
125-
126-
### Queue Service (port 8089)
26+
All Connect/gRPC services are mounted on a single port. Notable handlers:
12727

128-
- `POST /flyteidl2.workflow.QueueService/EnqueueAction` - Create TaskAction CR
129-
- `POST /flyteidl2.workflow.QueueService/AbortQueuedRun` - Delete root TaskAction
130-
- `POST /flyteidl2.workflow.QueueService/AbortQueuedAction` - Delete specific TaskAction
28+
- `flyteidl2.workflow.RunService` - Create / Get / List / Abort runs
29+
- `flyteidl2.workflow.InternalRunService` - Internal run-control APIs used by the executor
30+
- `flyteidl2.workflow.TranslatorService` - Translates user task definitions
31+
- `flyteidl2.workflow.RunLogsService` - Stream logs for a run
32+
- `flyteidl2.actions.ActionsService` - Action lifecycle and metadata
33+
- `flyteidl2.task.TaskService` - Task registration and lookup
34+
- `flyteidl2.trigger.TriggerService` - Schedules and triggers
35+
- `flyteidl2.project.ProjectService` - Project management
36+
- `flyteidl2.auth.IdentityService` / `AuthMetadataService` - Identity and auth metadata
37+
- DataProxy, Events, Cache, Secret, and App services (see their respective packages)
13138
- `GET /healthz` - Health check
13239
- `GET /readyz` - Readiness check
13340

@@ -138,26 +45,20 @@ logger:
13845

13946
## How It Works
14047

141-
1. **CreateRun** → Runs Service persists run to SQLite DB
142-
2. **CreateRun** → Runs Service calls Queue Service to enqueue root action
143-
3. **EnqueueAction** → Queue Service creates TaskAction CR in Kubernetes
144-
4. **Executor** → Watches TaskAction CRs and reconciles them
145-
5. **Executor** → Transitions: Queued → Initializing → Running → Succeeded
146-
6. **Executor** → Calls State Service Put() on each transition
147-
7. **State Service** → Persists state updates to SQLite DB
148-
8. **State Service** → Notifies watchers of state changes
48+
1. **CreateRun** → Runs Service persists the run to PostgreSQL and calls `ActionsService.Enqueue(...)` to enqueue the root action
49+
2. **Actions Service / Executor** → That enqueue flow results in the root TaskAction CR being created in Kubernetes, which the Executor then watches and reconciles
50+
3. **Executor** → Transitions: Queued → Initializing → Running → Succeeded
51+
4. **Actions Service** → Watches TaskAction CRs via a shared informer and forwards status updates (phase, output URI, error state) to subscribers; sdk controller consumes these updates through `WatchForUpdates` to drive the run forward
52+
5. **Runs Service** → Persists state changes to PostgreSQL and notifies its own watchers
14953

15054
## Testing
15155

15256
### Check Services
15357

15458
```bash
155-
# Runs Service health
59+
# Manager (Connect services) health
15660
curl http://localhost:8090/healthz
15761

158-
# Queue Service health
159-
curl http://localhost:8089/healthz
160-
16162
# Executor health
16263
curl http://localhost:8081/healthz
16364
```
@@ -178,11 +79,11 @@ kubectl describe taskaction <name> -n flyte
17879
### Check Database
17980

18081
```bash
181-
# Open SQLite database
182-
sqlite3 flyte.db
82+
# Connect to the PostgreSQL backend (devbox defaults)
83+
psql -h localhost -p 30001 -U postgres -d runs
18384

18485
# List tables
185-
.tables
86+
\dt
18687

18788
# Query runs
18889
SELECT * FROM runs;
@@ -199,55 +100,6 @@ SELECT name, phase, state FROM actions;
199100
make run
200101
```
201102

202-
### Docker
203-
204-
```dockerfile
205-
FROM golang:1.21 AS builder
206-
WORKDIR /app
207-
COPY . .
208-
RUN cd manager && make build
209-
210-
FROM alpine:latest
211-
RUN apk --no-cache add ca-certificates sqlite
212-
WORKDIR /root/
213-
COPY --from=builder /app/manager/bin/flyte-manager .
214-
COPY --from=builder /app/manager/config.yaml .
215-
CMD ["./flyte-manager", "--config", "config.yaml"]
216-
```
217-
218-
### Kubernetes
219-
220-
Deploy as a single pod with access to the Kubernetes API:
221-
222-
```yaml
223-
apiVersion: apps/v1
224-
kind: Deployment
225-
metadata:
226-
name: flyte-manager
227-
namespace: flyte
228-
spec:
229-
replicas: 1
230-
selector:
231-
matchLabels:
232-
app: flyte-manager
233-
template:
234-
metadata:
235-
labels:
236-
app: flyte-manager
237-
spec:
238-
serviceAccountName: flyte-manager
239-
containers:
240-
- name: manager
241-
image: flyte-manager:latest
242-
ports:
243-
- containerPort: 8090
244-
name: runs
245-
- containerPort: 8089
246-
name: queue
247-
- containerPort: 8081
248-
name: health
249-
```
250-
251103
## Troubleshooting
252104

253105
### Connection Issues
@@ -280,12 +132,11 @@ manager:
280132
```bash
281133
# Check what's using the ports
282134
lsof -i :8090
283-
lsof -i :8089
284135
lsof -i :8081
285136

286137
# Change ports in config.yaml
287138
manager:
288-
runsService:
139+
server:
289140
port: 9090 # Changed from 8090
290141
```
291142

‎manager/config.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ runs:
6565
postgres:
6666
host: "localhost"
6767
port: 30001
68-
dbName: "runs"
68+
dbname: "runs"
6969
username: "postgres"
7070
password: "postgres"
7171
options: "sslmode=disable"

0 commit comments

Comments
 (0)