Skip to content

Commit 43a5a78

Browse files
bsctlbsctl
committed
initial commit
0 parents  commit 43a5a78

19 files changed

+1030
-0
lines changed

‎.github/workflows/lint.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
name: Lint
2+
3+
on:
4+
push:
5+
branches: [ "*" ]
6+
pull_request:
7+
branches: [ "*" ]
8+
9+
jobs:
10+
lint:
11+
runs-on: ubuntu-latest
12+
steps:
13+
- name: Checkout code
14+
uses: actions/checkout@v4
15+
16+
- name: Set up Helm
17+
uses: azure/setup-helm@v4
18+
with:
19+
version: v3.3.4
20+
21+
- name: Linting Chart
22+
run: helm lint ./charts/capi-kamaji-proxmox

‎.github/workflows/release.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
tags: [ "v*" ]
6+
7+
jobs:
8+
release:
9+
runs-on: ubuntu-latest
10+
steps:
11+
- name: Checkout code
12+
uses: actions/checkout@v4
13+
14+
- name: Set up Helm
15+
uses: azure/setup-helm@v4
16+
with:
17+
version: v3.3.4
18+
19+
- name: Package Helm chart
20+
run: helm package ./charts/capi-kamaji-proxmox
21+
22+
- name: Publish Helm chart
23+
uses: stefanprodan/helm-gh-pages@master
24+
with:
25+
token: ${{ secrets.BOT_GITHUB_TOKEN }}
26+
charts_dir: charts
27+
charts_url: https://clastix.github.io/charts
28+
owner: clastix
29+
repository: charts
30+
branch: gh-pages
31+
target_dir: .
32+
commit_username: bsctl
33+
commit_email: bsctl@clastix.io

‎README.md‎

Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
# Cluster API Kamaji Proxmox Virtual Environment
2+
3+
This Helm chart deploys a Kubernetes cluster on Proxmox Virtual Environment using [Cluster API](https://cluster-api.sigs.k8s.io/) with [Kamaji](https://kamaji.clastix.io/) as the Control Plane Provider.
4+
5+
The Proxmox Cluster API implementation is initiated and maintained by [IONOS Cloud](https://github.com/ionos-cloud).
6+
7+
8+
## Table of Contents
9+
10+
- [Architecture Overview](#architecture-overview)
11+
- [Key Features](#key-features)
12+
- [Automatic Rolling Updates](#automatic-rolling-updates)
13+
- [Cluster Autoscaler Integration](#cluster-autoscaler-integration)
14+
- [Prerequisites](#prerequisites)
15+
- [Installation](#installation)
16+
- [Secret Management](#secret-management)
17+
- [Usage](#usage)
18+
- [Creating a cluster](#creating-a-cluster)
19+
- [Upgrading a cluster](#upgrading-a-cluster)
20+
- [Scaling a cluster](#scaling-a-cluster)
21+
- [Deleting a cluster](#deleting-a-cluster)
22+
- [Configuration](#configuration)
23+
- [Maintainers](#maintainers)
24+
- [Source Code](#source-code)
25+
- [License](#license)
26+
27+
## Architecture Overview
28+
29+
The chart implements a **Hosted Control Plane Architecture** where the Kubernetes control plane of the workload clusters run in the Management Cluster and are managed by the Kamaji Operator. This architecture allows for a more efficient use of resources and simplifies the management of the control plane components.
30+
31+
This approach also provides security benefits by isolating Proxmox VE credentials from tenant users while maintaining full Cluster API integration.
32+
33+
34+
## Key Features
35+
36+
### Automatic Rolling Updates
37+
38+
The chart supports seamless rolling updates of the workload clusters when the configuration changes. This works through Cluster API's machine lifecycle management for:
39+
40+
- Physical machine parameter changes, e.g. CPU, memory, disk
41+
- Kubernetes version upgrades
42+
- proxmox template changes
43+
- `cloud-init` configuration updates
44+
45+
The implementation uses hash-suffixed templates, `ProxmoxMachineTemplate` and `KubeadmConfigTemplate` that:
46+
1. Generate a new template with updated configuration and unique name on `helm upgrade`
47+
2. Update references in `MachineDeployment` to the new template
48+
3. Trigger Cluster API's built-in rolling update process
49+
50+
#### Rolling Update Workflow
51+
52+
1. Update `values.yaml` with new configuration
53+
2. Run: `helm upgrade my-cluster ./cluster-api-kamaji-proxmox`
54+
3. Cluster API automatically replaces nodes using the new configuration
55+
56+
57+
58+
### Cluster Autoscaler Integration
59+
60+
The chart includes support for enabling the Cluster Autoscaler for each node pool. This feature allows you to mark node pool machines to be autoscaled. However, you still need to install the Cluster Autoscaler separately.
61+
62+
The Cluster Autoscaler runs in the management cluster, following the hosted control plane model, and manages the scaling of the workload cluster. To enable autoscaling for a node pool, set the `autoscaling.enabled` field to `true` in your `values.yaml` file:
63+
64+
```yaml
65+
nodePools:
66+
- name: default
67+
replicas: 3
68+
autoscaling:
69+
enabled: true
70+
minSize: 2
71+
maxSize: 6
72+
labels:
73+
autoscaling: "enabled"
74+
```
75+
76+
This configuration marks the node pool for autoscaling. The Cluster Autoscaler will use these settings to scale the node pool within the specified limits.
77+
78+
You need to install the Cluster Autoscaler in the management cluster. Here is an example using Helm:
79+
80+
```bash
81+
helm repo add autoscaler https://kubernetes.github.io/autoscaler
82+
helm repo update
83+
helm upgrade --install ${CLUSTER_NAME}-autoscaler autoscaler/cluster-autoscaler \
84+
--set cloudProvider=clusterapi \
85+
--set autodiscovery.namespace=default \
86+
--set "autoDiscovery.labels[0].autoscaling=enabled" \
87+
--set clusterAPIKubeconfigSecret=${CLUSTER_NAME}-kubeconfig \
88+
--set clusterAPIMode=kubeconfig-incluster
89+
```
90+
91+
This command installs the Cluster Autoscaler and configures it to manage the workload cluster from the management cluster.
92+
93+
## Prerequisites
94+
95+
- Management Cluster with Kubernetes 1.28+
96+
- Kamaji installed and configured on the Management Cluster
97+
- Cluster API installed and configured on the Management Cluster with the [Proxmox Provider](https://github.com/ionos-cloud/cluster-api-provider-proxmox)
98+
- IPAM provider (optional)
99+
- Helm 3.x
100+
- Access to a Proxmox VE cluster with the necessary permissions to create and manage VMs
101+
102+
## Installation
103+
104+
```bash
105+
# Add repository (if published)
106+
helm repo add clastix https://clastix.github.io/charts
107+
helm repo update
108+
109+
# Install with custom values
110+
helm install my-cluster clastix/capi-kamaji-proxmox -f my-values.yaml
111+
```
112+
113+
## Secret Management
114+
115+
The chart requires a proxmox credential secret.
116+
117+
It is strongly recommended to use dedicated Proxmox VE user + API token:
118+
119+
```bash
120+
export PROXMOX_URL: "https://pve.example:8006" # The Proxmox VE host
121+
export PROXMOX_TOKEN: "clastix@pam!capi" # The Proxmox VE TokenID for authentication
122+
export PROXMOX_SECRET: "REDACTED" # The secret associated with the TokenID
123+
```
124+
125+
Then you can leave the chart to create the secret for you by filling the proper fields in your `values.yaml` or you can create the secret manually.
126+
127+
```bash
128+
# Create the proxmox secret for Cluster API
129+
cat <<EOF | kubectl apply -f -
130+
apiVersion: v1
131+
kind: Secret
132+
metadata:
133+
name: proxmox-secret
134+
labels:
135+
cluster.x-k8s.io/cluster-name: "my-cluster"
136+
stringData:
137+
url: "${PROXMOX_URL}"
138+
token: "${PROXMOX_TOKEN}"
139+
secret: "${PROXMOX_SECRET}"
140+
type: Opaque
141+
142+
EOF
143+
```
144+
145+
and reference it in your `values.yaml`:
146+
147+
```yaml
148+
# Using existing secrets
149+
proxmox:
150+
secret:
151+
create: false
152+
name: proxmox-secret
153+
```
154+
155+
## Usage
156+
157+
### Creating a cluster
158+
159+
```bash
160+
# Deploy using the chart
161+
helm install my-cluster clastix/capi-kamaji-proxmox -f values.yaml
162+
163+
# Check status
164+
kubectl get cluster,machines
165+
166+
# Get kubeconfig
167+
clusterctl get kubeconfig my-cluster > my-cluster.kubeconfig
168+
```
169+
170+
### Upgrading a cluster
171+
172+
```bash
173+
# Update values.yaml
174+
cluster:
175+
version: "v1.32.2"
176+
nodePools:
177+
- name: default
178+
templateId: 100
179+
180+
# Apply upgrade
181+
helm upgrade my-cluster clastix/capi-kamaji-proxmox -f values.yaml
182+
183+
# Watch the rolling update
184+
kubectl get machines -w
185+
```
186+
187+
### Scaling a cluster
188+
189+
```bash
190+
# Update values.yaml
191+
nodePools:
192+
- name: default
193+
replicas: 5
194+
195+
# Apply scaling
196+
helm upgrade my-cluster clastix/capi-kamaji-proxmox -f values.yaml
197+
198+
# Watch the scaling
199+
kubectl get machines -w
200+
```
201+
202+
### Deleting a cluster
203+
204+
```bash
205+
# Delete the cluster
206+
helm uninstall my-cluster
207+
```
208+
209+
## Configuration
210+
211+
See the [values](charts/capi-kamaji-proxmox/README.md) you can override.
212+
213+
## Maintainers
214+
215+
| Name | Email | Url |
216+
| ---- | ------ | --- |
217+
| Clastix Labs | <authors@clastix.labs> | |
218+
219+
## Source Code
220+
221+
* <https://github.com/clastix/cluster-api-kamaji-proxmox>
222+
223+
## License
224+
225+
This project is licensed under the Apache2 License. See the LICENSE file for more details.

‎charts/capi-kamaji-proxmox/Chart.yaml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: capi-kamaji-proxmox
2+
version: 0.0.1
3+
appVersion: 1.31.4
4+
description: A Helm chart for deploying a Kamaji Tenant Cluster on Proxmox VE using Cluster API
5+
and Kamaji.
6+
icon: https://cluster-api.sigs.k8s.io/images/introduction.svg
7+
type: application
8+
apiVersion: v2
9+
maintainers:
10+
- name: Clastix Labs
11+
email: authors@clastix.labs
12+
keywords:
13+
- kubernetes
14+
- helm
15+
- cluster-api
16+
- kamaji
17+
- proxmox
18+
home: https://kamaji.clastix.io
19+
sources:
20+
- https://github.com/clastix/cluster-api-kamaji-proxmox
21+
annotations:
22+
artifacthub.io/links: |
23+
- name: Clastix Labs
24+
url: https://github.com/clastix
25+
artifacthub.io/operator: "true"
26+
artifacthub.io/operatorCapabilities: "Full Lifecycle"
27+
artifacthub.io/changes: |
28+
- kind: commit
29+
description: Initial commit.

0 commit comments

Comments
 (0)