Skip to content

Latest commit

 

History

History
138 lines (118 loc) · 11 KB

File metadata and controls

138 lines (118 loc) · 11 KB

swarmpit

lightweight docker swarm management ui. clojure backend + clojurescript (rum/react) frontend, couchdb for app data, influxdb 1.x (optional) for stats. deployed via docker stack deploy. see docker-compose.yml for the reference stack: app + agent + db (couchdb) + influxdb:1.8.

build / test / dev

  • lein check — quick compile sanity gate, run after any clj edit
  • lein test — unit tests (:default selector excludes ^:integration)
  • lein test :integration — integration tests; need dind running (dev/script/start-dind.sh)
  • lein test :all — everything
  • lein uberjar — production jar (target/swarmpit.jar, entrypoint swarmpit.server/-main)
  • lein figwheel — dev: starts nrepl + figwheel, auto-reloads cljs on save. backend reloaded via ring-reload (so most handler changes hot-reload too). the figwheel bootstrap runs in dev/repl/user.clj: fig-start calls dev/script/init-db.sh, init-agent.sh, init-influx.sh to bring up supporting containers, then db/init, agent/init, setup/docker, setup/log.
  • repl-driven: (fig-start) / (fig-stop) / (cljs-repl) in repl.user.

build chain: cljsbuild with two builds — app (dev, figwheel-hooked) and min (advanced-compile for uberjar). :main is swarmpit.server. uberjar profile inherits prod which AOTs everything and runs cljsbuild once min.

source layout

src/clj/swarmpit/          backend
  api.clj                  top-level api fns (services, stacks, registries, users, …)
  handler.clj              ring handlers, resp-ok / resp-error helpers, login
  server.clj               -main, middleware stack, reitit router
  database.clj             boot init for couch + influx; influx wrapped in try/catch
  setup.clj                docker api version negotiation via /_ping
  config.clj               config atoms: @default ← environment ← @dynamic (later wins)
  version.clj              /api/version response body
  authorization.clj        access rules (buddy-auth, regex per path)
  agent.clj                schedules stats collection, event push
  stats.clj                stats collector; influx writes wrapped in try/catch
  slt.clj                  short-lived tokens for SSE auth
  docker/engine/
    http.clj               docker http client over unix socket, per-request conn-mgr
    client.clj             docker api methods; ping uses :unversioned? option
    cli.clj                shells out to docker cli for `stack deploy`, `stack rm`
    mapper/inbound.clj     docker spec → swarmpit domain
    mapper/outbound.clj    swarmpit domain → docker spec
    mapper/compose.clj     swarmpit domain → compose yaml
  couchdb/
    client.clj             couchdb http client
    migration.clj          run-once migrations, tracked in db
  influxdb/
    client.clj             influxdb 1.x client; queries raw influxql
  event/
    handler.clj            SSE endpoint via http-kit with-channel
    channel.clj            subscriber hub (atom of channel → request)
    processor.clj          event dispatch
  agent/                   agent client (used by service-agent-logs etc.)

src/cljs/swarmpit/         frontend (rum components)
  app.cljs                 entry
  ajax.cljs                wraps cljs-ajax, attaches Authorization: <jwt> from localStorage
  storage.cljs             localStorage wrappers
  event/source.cljs        js/EventSource client, auto-reconnects every 5s
  component/
    state.cljs             single app atom + cursors (docker-api-cursor, layout-cursor, form-value-cursor, …)
    layout.cljs            document-title + page routing switch
    menu.cljs              drawer / sidebar
    header.cljs            appbar
    common.cljs            title-logo, title-version, misc
    service/               service views (edit, info, form-*)
    stack/                 stack create / compose edit
    node/ network/ volume/ secret/ config/ registry/ user/

src/cljc/swarmpit/         shared
  routes.cljc              reitit routes; two top-level vars — `backend` (full reitit, with :handler) and `frontend` (plain [path name] pairs for cljs routing). helpers: path-for-backend, path-for-frontend.
  routes_spec.cljc         request/response schemas (spec-tools data-spec)
  yaml.cljc                yaml encode/decode (snakeyaml on jvm, js-yaml on cljs)
  utils.cljc               clean, clean-nils, name-value↔map

test/clj/swarmpit/         unit + ^:integration tests
resources/public/          css (main.css, ~1700 lines), fonts, img/{logo,icon}.svg
doc/                       configuration.md, USER_CONFIG.md, user_types.md
dev/script/                init-db.sh, init-agent.sh, init-influx.sh, start-dind.sh, …
dev/repl/user.clj          figwheel bootstrap (fig-start, cljs-repl)
.github/workflows/         build.yml (PR/push to master), release.yml (tag-driven)

domain model + non-obvious behaviour

  • service image string: build as repo:tag@digest (not repo@digest) so the tag stays visible in the ui even when digest-pinned. both redeploy-service in api.clj and ->service-image in mapper/outbound.clj must agree.
  • compose rendering walks the service map through utils/clean which recursively removes nil / empty collections. if a value becomes empty after cleaning, ->yaml emits {}\n. guard against that when adding a new renderer branch.
  • editing stacks goes through docker stack deploy cli (via docker/engine/cli.clj), not the service api. when a stored stackfile exists in couch, it's the source of truth; otherwise compose is reconstructed from the live docker state.
  • mounts: bind (has host path), volume (host is volume name; may be named or anonymous), tmpfs (no host, target only). request specs treat :host as optional on mount payloads.
  • ports: short syntax "host:container" when mode=ingress and protocol=tcp; long syntax (target/published/protocol/mode) otherwise.
  • influxdb: 1.x only. boot wraps init in try/catch, runtime writes wrap in try/catch → a broken or v2 influx degrades silently to "no stats", swarmpit still runs. reference compose uses influxdb:1.8.
  • docker api version: default 1.44, negotiated at boot via /_ping clamped to our max. SWARMPIT_DOCKER_API overrides.
  • entrypoint vs command naming confusion: compose entrypoint ↔ docker ContainerSpec.Command, compose command ↔ docker ContainerSpec.Args. remember whenever you touch the mappers.
  • containerLabels vs labels are separate: containerLabelsContainerSpec.Labels (per-container), labels → top-level Spec.Labels (per-service). both round-trip.
  • swarmpit-specific docker labels (on services swarmpit deploys/manages):
    • swarmpit.service.deployment.autoredeploy — bool, drives auto-redeploy on new digest
    • swarmpit.service.immutable — bool, disables edits in ui
    • swarmpit.agent — bool, service is a swarmpit agent
    • swarmpit.service.link.<name> — user-defined link, key/value stored as label
  • memoization: clojure.core.memoize/ttl used to coalesce request bursts. inherited caches:
    • tasks-memo, services-memo, admin-exists? — 1s ttl
    • df-memo — 10s ttl
    • broadcast-memo (events) — 1s ttl if a change "doesn't show up", it's often a cache — wait up to the ttl or restart.
  • couchdb migrations: map in couchdb/migration.clj under migrations. ran once, tracked in db; to add one, register a new keyword + fn. never mutate existing migrations in place.

frontend conventions

  • state lives in one atom (swarmpit.component.state/state); access via cursors (docker-api-cursor, docker-engine-cursor, instance-name-cursor, form-value-cursor, form-state-cursor, layout-cursor). react via state/react inside rum/defc.
  • on page load version-handler (component/layout.cljs) fetches /api/version and (state/set-value response) merges the response at the root — so :docker, :initialized, :instanceName sit at top level and are readable via single-key cursors.
  • auth: POST /login with Authorization: Basic <base64> returns {:token <jwt>}. frontend persists in localStorage via swarmpit.storage; swarmpit.ajax attaches Authorization: <raw-jwt> on every request (no Bearer prefix — intentional, matches backend parser).
  • SSE: /events uses http-kit with-channel, returns text/event-stream. client (event/source.cljs) uses js/EventSource; reconnects every 5s on error. SSE auth uses a short-lived token (/slt), not the JWT, because EventSource can't set headers.
  • material-ui v4 via @material-ui/core. popper modifiers follow popper.js v1 shape ({:flip {:enabled false}}, not v2 array form).
  • css naming: .Swarmpit-<area>-<element>. dark-mode variants live further down the same file under html.dark .Swarmpit-....
  • sablono [:<>] fragments inside rum/defc are unreliable — wrap in a [:div].
  • rum/defc does not support multi-arity; use default args via a plain fn wrapper.

backend conventions

  • middleware stack (server.clj, in order): muuntaja format → authentication → authorization → swagger → parameters → exception → request-coerce ON, response-coerce OFF. so responses don't get schema-stripped; request bodies do.
  • authorization rules (authorization.clj): regex + method, buddy.auth.accessrules. baseline is ^/api/.* → authenticated, with narrower rules for admin endpoints, per-registry ownership, etc.
  • the default exception handler switches on :type in ex-data (:http-client, :aws-client, :docker-cli, :api) to produce useful bodies; unmatched exceptions become {:status 500 :body (Throwable->map e)}.
  • reitit specs don't hot-reload: edits to routes_spec.cljc require a full jvm restart — the router is compiled at startup. repl fig-stop + fig-start rebinds it.
  • service logs: agent path preferred (service-agent-logs), falls back to native docker /logs (service-native-logs). detected via swarmpit.agent task label.

env vars

see doc/configuration.md. commonly touched:

  • SWARMPIT_DOCKER_API — override negotiated api version (otherwise auto-negotiated via /_ping)
  • SWARMPIT_DOCKER_SOCK/var/run/docker.sock or http://host:2375
  • SWARMPIT_DOCKER_HTTP_TIMEOUT — ms, default 5000
  • SWARMPIT_DB — couchdb url
  • SWARMPIT_INFLUXDB — stats url; nil disables stats (v2 incompatible — use 1.x)
  • SWARMPIT_AGENT_URL — static agent address, otherwise discovered dynamically; dev-only
  • SWARMPIT_WORK_DIR — temp dir, default /tmp
  • SWARMPIT_INSTANCE_NAME — custom sidebar/header/tab title (replaces swarmpit wordmark)

ci / release

  • .github/workflows/build.yml: on push/PR to master/develop. Builds uberjar, pushes multi-arch docker image to GHCR (ghcr.io/changemakerstudios/swarmpit) tagged with GitVersion-computed semver, plus :latest on master. multi-arch build (amd64/arm64/armv7/armv5).
  • .github/workflows/release.yml: triggered by version tag. builds, pushes tagged image, creates github release with jar.
  • uberjar is the ci artifact; docker stage only copies it and sets the entrypoint.