OKDP Sandbox is a hands-on environment for deploying, testing, and exploring the OKDP ecosystem on a local Kubernetes cluster.

It provides a ready-to-use data platform environment covering identity, object storage, Spark processing, workflow orchestration, interactive notebooks, SQL querying, and visualization, running end to end on a local cluster. See What is included for the full component list.

What is included in the sandbox?

The default OKDP sandbox deploys a complete local data-platform environment with:

• Keycloak for identity and access management
• SeaweedFS for object storage
• Spark Operator and Spark History Server for Spark workloads and monitoring
• Apache Airflow for workflow orchestration
• JupyterHub for interactive data-science workspaces
• Trino and Hive Metastore for SQL query and metadata services
• Apache Superset for dashboards and visual analytics
• OKDP Server and OKDP UI for platform management

Repository scope

This repository owns the single-cluster sandbox deployment. It describes how to deploy the OKDP platform onto a local Kubernetes cluster and contains the deployment assets only:

• clusters/sandbox/flux/ : Flux bootstrap of the KuboCD controller (kubocd.yaml)
• clusters/sandbox/releases/ : KuboCD Release manifests (what gets installed, which package tag, which parameters)
• clusters/sandbox/contexts/ : the layered KuboCD Context files (10-platform, 20-provider, 30-service, 99-examples)
• docs/ : deployment guides (DNS, certificates)

The packages themselves (the KuboCD packages bundled as OCI artifacts) live in a dedicated repository and are consumed here from the registry. This repository never builds packages, it only deploys published ones.

Concern	Owner
KuboCD packages (system and services), build and release automation	OKDP/platform-packages
Reusable utility Helm charts	OKDP/helm-charts-utilities
Notebooks, DAGs, and runnable examples	OKDP/okdp-examples
OKDP web UI	OKDP/okdp-ui
OKDP backend server (API)	OKDP/okdp-server
Single-cluster sandbox deployment (this repository)	OKDP/okdp-sandbox

Prerequisites

System requirements

• Minimum: 16 GB RAM and 4 CPUs
• Docker or Podman allocation: at least 8 GB RAM and 2 CPUs

Software dependencies

• Docker or a compatible container runtime
• Kind
• kubectl
• Flux CLI v2.7.5

Quick start

The sandbox deployment files live in this repository under clusters/sandbox/.

1. Clone this repository

git clone https://github.com/OKDP/okdp-sandbox.git
cd okdp-sandbox

2. Create Kubernetes Kind Cluster

Create a Kind cluster configuration file and deploy the cluster:

ℹ️ Kind is a tool for running local Kubernetes clusters using Docker.
It’s ideal for development, testing, and sandbox reproducible environments.
Kind follows a manifest-first (infrastructure-as-code) approach, while Minikube is a command-line-first approach.

# Create cluster configuration
cat > /tmp/okdp-sandbox-config.yaml <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: okdp-sandbox
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 30080
    hostPort: 80
  - containerPort: 30443
    hostPort: 443
  - containerPort: 30053
    hostPort: 30053
    protocol: UDP
EOF

# Create the cluster
kind create cluster --config /tmp/okdp-sandbox-config.yaml

PowerShell

# Create cluster configuration
@"
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: okdp-sandbox
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 30080
    hostPort: 80
  - containerPort: 30443
    hostPort: 443
  - containerPort: 30053
    hostPort: 53
    protocol: UDP
"@ | Out-File -FilePath "$env:TEMP\okdp-sandbox-config.yaml" -Encoding UTF8

# Create the cluster
kind create cluster --config "$env:TEMP\okdp-sandbox-config.yaml"

3. Install Platform Components

Install Flux (GitOps engine)

ℹ️ Note
This step is only required for a fresh installation. It is not required for upgrades if Flux is already installed and running.
For upgrades, jump to Deploy/Upgrade OKDP platform components.

ℹ️ Flux is the GitOps controller that continuously reconciles your cluster state with what’s defined in Git.
The following command installs all Flux core components:

• source-controller: fetches sources such as Git repositories and Helm charts
• kustomize-controller: applies Kubernetes manifests using Kustomize
• helm-controller: manages Helm releases declaratively
• notification-controller: handles alerts and automation triggers

💡 In this setup, Flux controllers manage resources locally and are not connected to a Git repository.
Manifests are applied manually with kubectl, so no Git access is required.

flux install

Configure proxy settings for Flux controllers (Optional)

If your environment requires a proxy to reach external sources (container registries), the following command sets the proxy configuration variables to all Flux controllers (source, kustomize, helm, notification):

[ -n "${https_proxy}${HTTPS_PROXY}" ] && kubectl -n flux-system set env deploy -l app.kubernetes.io/part-of=flux \
        HTTPS_PROXY="${HTTPS_PROXY:-${https_proxy}}" \
        HTTP_PROXY="${HTTP_PROXY:-${http_proxy}}" \
        NO_PROXY="${NO_PROXY:-${no_proxy}}"

PowerShell

if ($env:HTTPS_PROXY -or $env:https_proxy) {
    kubectl -n flux-system set env deploy -l app.kubernetes.io/part-of=flux `
        HTTPS_PROXY=($env:HTTPS_PROXY ?? $env:https_proxy) `
        HTTP_PROXY=($env:HTTP_PROXY ?? $env:http_proxy) `
        NO_PROXY=($env:NO_PROXY ?? $env:no_proxy)
}

Verify the proxy environment variables are correctly set for all Flux controllers:

💡 You may see the same variable (e.g., HTTPS_PROXY) repeated multiple times, one for each controller (source, kustomize, helm, notification).
This is expected and confirms that the variables were applied consistently.

kubectl -n flux-system set env deploy -l app.kubernetes.io/part-of=flux --list \
                                         | grep PROXY

💡 Use the following command if you want to remove the proxy configuration from Flux controllers:
After removing the proxy, Flux will no longer be able to pull images or manifests from external registries that require proxy access.

kubectl -n flux-system set env deploy -l app.kubernetes.io/part-of=flux \
   HTTPS_PROXY- \
   NO_PROXY-

Wait for Flux controllers to be ready

Ensures all Flux controllers (source-controller, kustomize-controller, helm-controller, notification-controller) are fully running before proceeding to the next step:

kubectl wait --for=condition=ready pod -l app=source-controller -n flux-system --timeout=300s

Install KuboCD (Flux extension)

ℹ️ KuboCD is the continuous delivery layer built on top of Flux.
It manages platform components and applications declaratively, providing a higher-level CD abstraction for GitOps workflows.

kubectl apply -f clusters/sandbox/flux/kubocd.yaml

Deploy/Upgrade OKDP platform components

ℹ️ Note
To upgrade the OKDP platform components, run:

kubectl delete $(kubectl get release -n kubocd-system -o name) -n kubocd-system

This will delete all KuboCD Release resources in the kubocd-system namespace.

During upgrade command, you may see errors like:

Error from server (Forbidden): admission webhook "vrelease-v1alpha1.kb.io" denied the request: release cert-manager is protected
Error from server (Forbidden): admission webhook "vrelease-v1alpha1.kb.io" denied the request: release kubocd-webhooks is protected

These errors can be safely ignored. The affected releases are system-protected components managed by the platform and have a separate upgrade lifecycle.

Pull the latest updates locally before starting the upgrade.

git pull --rebase

Deploy/Upgrade the sandbox default context:

💡 The KuboCD Context is a centralized, reusable, declarative and environment-aware configuration layer that provides user defined shared parameters (ingress suffixes, storage classes, certificate issuers, catalogs, and authentication settings, etc) to all the components, ensuring consistent deployment.

During deployment, KuboCD automatically resolves and injects these context variables into the target Kubernetes components across the cluster (cluster-wide), ensuring that every component is deployed with a consistent configuration.

During a Context update, changes are automatically propagated only to the affected components, which are then reconciled to align with the desired configuration.

For example, the Context enables defining different configurations for different environments:

• sandbox for experimentation
• dev for internal testing
• prod for stable production environments
• org (or global) for the organization-wide configuration that provides defaults to other environments.

Each environment can define, override or extend one or more contexts while preserving a unified, declarative deployment model.

kubectl apply -f clusters/sandbox/contexts/10-platform-context.yaml
kubectl apply -f clusters/sandbox/contexts/20-provider-context.yaml
kubectl apply -f clusters/sandbox/contexts/30-service-context.yaml
kubectl apply -f clusters/sandbox/contexts/99-examples-context.yaml

💡 By default, the default Context uses okdp.sandbox as the ingress domain suffix.
This domain may be blocked if it does not comply with your organization’s allowed domain policy.

Use the following command to update the domain suffix to match your organization’s domain (replace <CUSTOM_DOMAIN> with your actual domain name):

kubectl -n kubocd-system patch context platform \
  -p '{"spec":{"context":{"platform":{"ingress":{"suffix":"<CUSTOM_DOMAIN>"}}}}}' \
  --type=merge

Configure proxy settings for OKDP Services (Optional)

If your environment requires a proxy to reach external datasets (Superset examples, okdp examples, quay.io KuboCD packages), the following command sets the proxy configuration variables to the required OKDP services:

kubectl -n kubocd-system patch context platform --type merge -p "$(cat <<EOF
spec:
  context:
    platform:
      proxy:
        httpProxy: "${HTTP_PROXY:-${http_proxy}}"
        httpsProxy: "${HTTPS_PROXY:-${https_proxy}}"
        noProxy: "${NO_PROXY:-${no_proxy}}"
EOF
)"

PowerShell

kubectl -n kubocd-system patch context platform --type merge -p @"
spec:
  context:
    platform:
      proxy:
        httpProxy: "$($env:HTTP_PROXY ?? $env:http_proxy)"
        httpsProxy: "$($env:HTTPS_PROXY ?? $env:https_proxy)"
        noProxy: "$($env:NO_PROXY ?? $env:no_proxy)"
"@

Deploy/Upgrade OKDP components:

kubectl apply -f clusters/sandbox/releases/

Verify and monitor release deployment status

Watch releases as they are deployed until all the components become ready.

kubectl get releases -A --watch
# Wait until all releases show STATUS=READY (press Ctrl+C to exit watch)
# Alternative: kubectl wait --for=condition=ready release --all --all-namespaces --timeout=600s

4. DNS Setup

Enable access to OKDP services through DNS resolution for the okdp.sandbox or your custom domain <CUSTOM_DOMAIN>:

• Option 1 (Recommended): Local DNS server configuration (recommended, automatic for all services)
• Option 2: Manual /etc/hosts configuration (simple but requires manual updates)

📋 See dns-configuration.md for detailed setup instructions for your operating system.

5. SSL Certificate

For HTTPS access without warnings, two options:

Option 1: Install the CA certificate

The sandbox uses a local certificate authority.

To avoid browser warnings, export the generated CA certificate and import it into your system or browser trust store:

kubectl get secret default-issuer -n cert-manager -o jsonpath='{.data.ca\.crt}' | base64 -d > okdp-sandbox-ca.crt

PowerShell

# Import okdp-sandbox-ca.crt into your system's or browser's certificate store
kubectl get secret default-issuer -n cert-manager -o jsonpath='{.data.ca\.crt}' | ForEach-Object { [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($_)) } | Out-File -FilePath "okdp-sandbox-ca.crt" -Encoding ASCII

Option 2: Ignore certificate warnings

• First, connect to Keycloak (https://keycloak.okdp.sandbox or https://keycloak.<CUSTOM_DOMAIN>) and accept the self-signed certificate in your browser.
• This step is mandatory for all OKDP services (UI, Seaweedfs, etc.) to communicate properly with Keycloak.

Quick Start Guide

1. Access OKDP UI: https://okdp-ui.okdp.sandbox or https://okdp-ui.<CUSTOM_DOMAIN>
2. Login credentials: Default authentication via Keycloak (login/password: adm/adm)
3. Run the examples: https://github.com/OKDP/okdp-examples

Cleanup

kind delete cluster --name okdp-sandbox
rm /tmp/okdp-sandbox-config.yaml

PowerShell

kind delete cluster --name okdp-sandbox
Remove-Item "$env:TEMP\okdp-sandbox-config.yaml" -Force

License

This project is licensed under the Apache License 2.0.

---

Built 🚀 for the OKDP Community