REDMANE Monorepo

This repository contains everything you need to run the REDMANE Data Registry application stack via Docker.

Overview

REDMANE (Research Data Management and Analysis Environment) is a web application that tracks research datasets, project data, etc. without actually storing them. This repo provides the top-level Docker configuration that ties together the frontend, backend, and database services with SSL termination via https-portal container (instead of separate nginx and certbot instances).

Architecture

┌─────────────────────────────────────────────────────────────────┐
│  Docker Network (redmane-network)                               │
│                                                                 │
│  ┌──────────────────┐                                           │
│  │  frontend-build  │                                           │
│  │  (Node.js)       │──► Builds React static files once,        │
│  │                  │    copies to shared volume, then exits    │
│  └──────────────────┘                                           │
│           │ frontend_static volume                              │
│           ▼                                                     │
│  ┌─────────────┐          ┌─────────────┐                       │
│  │ https-portal│          │  backend    │                       │
│  │  :80/:443   │─────────►│  (FastAPI)  │                       │
│  │             │ /fastapi/│  :8888      │                       │
│  │ Serves      │          └─────────────┘                       │
│  │ static      │                │                               │
│  │ React files │                │                               │
│  │ at /        │                ▼                               │
│  └─────────────┘          ┌─────────────┐                       │
│                           │     db      │                       │
│                           │ (PostgreSQL)│                       │
│                           │  :5432      │                       │
│                           └─────────────┘                       │
│                                                                 │
│  Path-based routing:                                            │
│    /           → Static React files (served by https-portal)    │
│    /fastapi/   → backend:8888                                   │
└─────────────────────────────────────────────────────────────────┘

Repository Structure

REDMANE_Docker/
├── docker-compose.yaml              # Main orchestration file
├── redmane_fastapi.dockerfile       # Backend container build
├── redmane_reactjs.dockerfile       # Frontend multi-stage build (build React, copy static files)
├── data-registry.wehi-rcp.cloud.edu.au.conf.erb  # Custom nginx config for path-based routing
├── .dockerignore                    # Excludes unnecessary files from Docker build context
├── .gitignore                       # Excludes files/folders that may cause conflict when cloning
└── README.md

Note: The following folders are cloned via git submodules:

• REDMANE_fastapi/ - Backend application (separate repo)
• REDMANE_react.js/ - Frontend application (separate repo)
• backups/ - VM's backup folder (not currently tracked in any git repo)

Related Repositories

• REDMANE_fastapi - FastAPI backend
• REDMANE_react.js - React frontend
• REDMANE_fastapi_public_data - Database initialization scripts
• REDMANE Metadata File Creation - creating file metadata to upload to Data Registry

Prerequisites

• A container runtime. One of:
  • Docker and Docker Compose installed (done)
  • Podman and Podman/Docker compose installed
• Access to the target VM (via SSH, check technical diaries)
• Domain DNS configured to point to the VM's IP address (done)

Context on https-portal

It is a ready-to-use docker image that combines the functionality of nginx (reverse proxying, hosting static frontend files, SSL termination) and certbot (automated SSL certificate obtainment and renewal/management). For more context, visit: https://github.com/SteveLTN/https-portal

Running locally

1. Clone the repository and all submodules:

git clone --recurse-submodules git@github.com:WEHI-ResearchComputing/REDMANE_monorepo.git .

2. Start the stack (podman)

cd REDMANE_monorepo
podman compose up --build -d

Access the running application at: http://localhost:8000/projects

Deployment Steps

1. Clone this repository on the REDMANE folder of the Data Registry VM

cd REDMANE
git clone --recurse-submodules git@github.com:WEHI-ResearchComputing/REDMANE_monorepo.git

2. Configure environment

Before starting, verify the following in docker-compose.yaml and environment files (.prod.env):

• STAGE is set appropriately (local, staging, or production)
  • Do not use 'production' unless the setup has been successfully tested in staging. 'production' will hit the production side of Let's Encrypt and they have rate limits
• Database credentials match other parts of the backend and database codes
• Domain name in https-portal container's DOMAINS variable matches your setup

3. Start the stack (production)

docker compose --env-file .prod.env up --build -d

The frontend-build container will build the React application, copy the static files to a shared volume, and then exit. This is expected — docker ps -a will show it as "Exited (0)".

5. Verify deployment

• Frontend: access https://data-registry.wehi-rcp.cloud.edu.au/projects on an incognito browser (so that caches are not saved and you can test builds more accurately)
• Backend API: perform this on your terminal:

curl https://data-registry.wehi-rcp.cloud.edu.au/projects # if SSL certs have been obtained
curl -k https://data-registry.wehi-rcp.cloud.edu.au/projects # in local/staging STAGE

Configuration Details

https-portal Stages

Stage	Purpose	Certificates
local	Local development	Self-signed
staging	Testing SSL setup	Let's Encrypt staging (not trusted)
production	Live deployment	Let's Encrypt production (trusted)

Important: Always test with staging before switching to production to avoid hitting Let's Encrypt rate limits.

Path-Based Routing

The .conf.erb file configures nginx (inside https-portal) to:

• Serve static React files at / directly from the shared volume (with try_files fallback to index.html for client-side routing)
• Proxy all requests to /fastapi/ → FastAPI backend container

Frontend Multi-Stage Build

The redmane_reactjs.dockerfile uses a two-stage Docker build:

1. Stage 1 (Node.js): Installs dependencies and runs npm run build to produce static files in dist/
2. Stage 2 (Alpine): Copies only the built dist/ files into a lightweight image (~5MB), discarding Node.js and node_modules

At runtime, the container copies the static files to a shared Docker volume (frontend_static) that https-portal serves directly. The container then exits.

Troubleshooting

Check container logs

docker compose logs -f [service_name]

Rebuild after changes

docker compose down
docker compose --env-file .<env>.env up --build -d

Reset the database to default seed data

There are two methods to reset the database.

Method 1: Run the reset script (recommended — no downtime for containers)

bash reset_database.sh

This script drops the public schema, recreates it, and re-runs the init SQL file, all while keeping the Docker containers running. See reset_database.sh for details on what it does.

Method 2: Remove the PostgreSQL volume (nuclear option — requires restarting the stack)

# Backup current database first (optional but recommended)
docker exec redmane-db pg_dump -U postgres readmedatabase > ~/REDMANE/backups/database_backup_$(date +%Y%m%d_%H%M%S).sql

# Stop all containers
docker compose down

# Remove the postgres volume to trigger re-initialisation
docker volume rm redmane_postgres_data

# Bring everything back up
docker compose --env-file .<env>.env up -d

This forces PostgreSQL to re-run the init scripts on startup since the data volume is empty.