[WIP] feat: replace uwsgi with granian

docs/configuration.rst

@@ -140,10 +140,10 @@ This defines the default version that will be pulled from all Open edX git repos
 
 This defines the version that will be pulled from just the Open edX platform git repositories. You may also override this configuration parameter at build time, by providing a ``--build-arg`` option.
 
-- ``OPENEDX_CMS_UWSGI_WORKERS`` (default: ``2``)
-- ``OPENEDX_LMS_UWSGI_WORKERS`` (default: ``2``)
+- ``OPENEDX_CMS_GRANIAN_WORKERS`` (default: ``2``)
+- ``OPENEDX_LMS_GRANIAN_WORKERS`` (default: ``2``)
 
-By default, there are 2 `uwsgi worker processes <https://uwsgi-docs.readthedocs.io/en/latest/Options.html#processes>`__ to serve requests for the LMS and the CMS. However, each worker requires upwards of 500 Mb of RAM. You should reduce this value to 1 if your computer/server does not have enough memory.
+By default, there are 2 Granian worker processes to serve requests for the LMS and the CMS. You should reduce this value to 1 if your computer/server does not have enough memory. You can increase it when you need higher throughput, as long as CPU and memory resources are available.
 
 - ``OPENEDX_CELERY_REDIS_DB`` (default: ``0``)
 - ``OPENEDX_CACHE_REDIS_DB`` (default: ``1``)

docs/local.rst

@@ -206,9 +206,9 @@ To update the course search index, run::
 Reloading Open edX settings
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-After modifying Open edX settings, for instance when running ``tutor config save``, you will want to restart the web processes of the LMS and the CMS to take into account those new settings. It is possible to simply restart the whole platform (with ``tutor local reboot``) or just a single service (``tutor local restart lms``) but that is overkill. A quicker alternative is to send the HUP signal to the uwsgi processes running inside the containers. The "openedx" Docker image comes with a convenient script that does just that. To run it, execute::
+After modifying Open edX settings, for instance when running ``tutor config save``, you will want to restart the web processes of the LMS and the CMS to take into account those new settings. It is possible to simply restart the whole platform (with ``tutor local reboot``) or just a single service (``tutor local restart lms``) but that is overkill. A quicker alternative is to restart just the web server process running inside the containers. This can be done by updating a file in the granian's "--reload-paths" config. The "openedx" Docker image comes with a convenient script that does just that. To run it, execute::
 
-    tutor local exec lms reload-uwsgi
+    tutor local exec lms reload-granian
 
 
 Customizing the deployed services

docs/reference/patches/index.rst

@@ -400,13 +400,13 @@ File: ``apps/redis/redis.conf``
 
 Implement this patch to override hard-coded Redis configuration values. See the `Redis configuration reference <https://redis.io/docs/management/config-file/>`__`.
 
-``uwsgi-config``
-================
+.. patch:: granian-config
 
-File: ``apps/openedx/settings/uwsgi.ini``
+``granian-config``
+==================
 
-A .INI formatted file used to extend or override the uWSGI configuration.
+File: ``apps/openedx/settings/granian-config``
 
-Check the uWSGI documentation for more details about the `.INI format <https://uwsgi-docs.readthedocs.io/en/latest/Configuration.html#ini-files>`__ and the `list of available options <https://uwsgi-docs.readthedocs.io/en/latest/Options.html>`__.
+A file used to extend or override the Granian configuration.
 
-.. patch:: uwsgi-config
+Check the Granian documentation for the `list of available options <https://github.com/emmett-framework/granian>`__.

docs/sysadmin/proxy.rst

@@ -11,7 +11,7 @@ Quite often, there is already a web proxy running on the host, and this web prox
 
 With these changes, Tutor will no longer listen to ports 80 and 443 on the host. In this configuration, the Caddy container will only listen to port 81 on the host. Web requests will follow this path::
 
-    Client → Web proxy (http(s)://yourhost) → Caddy (0.0.0.0:81) → uwsgi (LMS/CMS/...)
+    Client → Web proxy (http(s)://yourhost) → Caddy (0.0.0.0:81) → Granian (LMS/CMS/...)
 
 .. warning::
     In this setup, the Caddy HTTP port (81) will be exposed to the world. Make sure to configure your server firewall to block unwanted connections to the Caddy container. Alternatively, you can configure the Caddy container to accept only local connections::

docs/sysadmin/scale.rst

@@ -23,11 +23,11 @@ Fortunately, Open edX was designed to run at scale -- most notably at `edX.org <
 Increasing web server capacity
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-As the server CPU and memory are increased, the request throughput can be increased by adjusting the number of uWSGI workers (see :ref:`configuration docs <openedx_configuration>`). By default, the "lms" and "cms" containers each spawn 2 uWSGI workers. The number of workers should be increased if you observe an increase in the latency of user requests but CPU usage remains below 100%. To increase the number of workers for the LMS and the CMS, run for example::
+As the server CPU and memory are increased, the request throughput can be increased by adjusting the number of Granian workers (see :ref:`configuration docs <openedx_configuration>`). By default, the "lms" and "cms" containers each spawn 2 Granian workers. The number of workers should be increased if you observe an increase in the latency of user requests but CPU usage remains below 100%. To increase the number of workers for the LMS and the CMS, run for example::
 
     tutor config save \
-      --set OPENEDX_LMS_UWSGI_WORKERS=8 \
-      --set OPENEDX_CMS_UWSGI_WORKERS=4
+      --set OPENEDX_LMS_GRANIAN_WORKERS=8 \
+      --set OPENEDX_CMS_GRANIAN_WORKERS=4
     tutor local restart lms cms
 
 The right values will very much depend on your server's available memory and CPU performance, as well as the maximum number of simultaneous users who use your platform. As an example data point, it was reported that a large Open edX platform can serve up to 500k unique users per week on a virtual server with 8 vCPU and 16 GB memory.

tutor/templates/apps/openedx/granian-config

@@ -0,0 +1,2 @@
+{% include "build/openedx/settings/granian-config" %}
+{{ patch("granian-config") }}

tutor/templates/build/openedx/Dockerfile

@@ -113,13 +113,10 @@ RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \
     # Use redis as a django cache https://pypi.org/project/django-redis/
     django-redis==5.4.0
 
-# uwsgi server https://pypi.org/project/uWSGI/
-# We don't need xml configuration support in uwsgi so don't install it, as it causes
-# uwsgi to crash
-# https://github.com/xmlsec/python-xmlsec/issues/320
+# granian server https://pypi.org/project/granian/
+# We need the reload dependency so install it as well
 RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared \
-    UWSGI_PROFILE_OVERRIDE="xml=no" \
-    $PIP_COMMAND install --no-cache-dir --compile uwsgi==2.0.24
+    $PIP_COMMAND install --no-cache-dir --compile granian[reload]==2.7.4
 
 # Install scorm xblock
 RUN $PIP_COMMAND install "openedx-scorm-xblock>=19.0.0,<20.0.0"
@@ -317,14 +314,20 @@ CMD ["./manage.py", "$SERVICE_VARIANT", "runserver", "0.0.0.0:8000"]
 ###### Final image with production cmd
 FROM production AS final
 
-# Default amount of uWSGI processes
-ENV UWSGI_WORKERS=2
+# Default amount of Granian worker processes
+ENV GRANIAN_WORKERS=2
 
-# Copy the default uWSGI configuration
-COPY --chown=app:app settings/uwsgi.ini /openedx
+# Copy granian configuration file
+COPY --chown=app:app settings/granian-config /openedx
+
+# Create the reload marker file
+RUN touch /openedx/granian-reload-path \
+    && chown app:app /openedx/granian-reload-path
 
 # Run server
-CMD ["uwsgi", "/openedx/uwsgi.ini"]
+# Note that we wrap the granian command inside python so that it can use a config file. 
+# Granian does not support config files out of the box. 
+CMD ["run-granian"]
 
 {{ patch("openedx-dockerfile-final") }}
 

tutor/templates/build/openedx/bin/reload-granian

@@ -0,0 +1,3 @@
+#! /bin/bash
+echo "Reloading granian process..."
+date +%s > /openedx/granian-reload-path

tutor/templates/build/openedx/bin/run-granian

@@ -0,0 +1,33 @@
+#!/usr/bin/env python3
+import os
+from granian.cli import cli
+
+
+def parse_args_file() -> list[str]:
+    args = []
+    with open("/openedx/granian-config") as f:
+        for line in f:
+            line = line.strip()
+            if not line or line.startswith("#"):
+                # skip empty lines and comments
+                continue
+
+            # Expand environment variables like $VAR_NAME
+            line = os.path.expandvars(line)
+
+            # Split by first space (for options with values)
+            if " " in line:
+                option, value = line.split(" ", 1)
+                args.append(option)
+                args.append(value)
+            else:
+                # flags without values
+                args.append(line)
+    return args
+
+
+def run():
+    cli(args=parse_args_file(), standalone_mode=False)
+
+if __name__ == "__main__":
+    run()

tutor/templates/build/openedx/settings/granian-config

@@ -0,0 +1,12 @@
+--interface wsgi
+--host 0.0.0.0
+--port 8000
+--workers $GRANIAN_WORKERS
+--static-path-route /static
+--static-path-mount /openedx/staticfiles/
+--static-path-route /media
+--static-path-mount /openedx/media/
+--reload
+--reload-paths /openedx/granian-reload-path
+--http1-buffer-size 8192
+$SERVICE_VARIANT.wsgi:application

tutor/templates/config/defaults.yml

@@ -56,8 +56,8 @@ OPENEDX_AWS_ACCESS_KEY: ""
 OPENEDX_AWS_SECRET_ACCESS_KEY: ""
 OPENEDX_CACHE_REDIS_DB: 1
 OPENEDX_CELERY_REDIS_DB: 0
-OPENEDX_CMS_UWSGI_WORKERS: 2
-OPENEDX_LMS_UWSGI_WORKERS: 2
+OPENEDX_CMS_GRANIAN_WORKERS: 2
+OPENEDX_LMS_GRANIAN_WORKERS: 2
 OPENEDX_MYSQL_DATABASE: "openedx"
 OPENEDX_MYSQL_USERNAME: "openedx"
 # the common version will be automatically set to "master" in the main branch

tutor/templates/k8s/deployments.yml

@@ -84,8 +84,8 @@ spec:
             value: cms
           - name: DJANGO_SETTINGS_MODULE
             value: cms.envs.tutor.production
-          - name: UWSGI_WORKERS
-            value: "{{ OPENEDX_CMS_UWSGI_WORKERS }}"
+          - name: GRANIAN_WORKERS
+            value: "{{ OPENEDX_CMS_GRANIAN_WORKERS }}"
           ports:
             - containerPort: 8000
           volumeMounts:
@@ -95,9 +95,9 @@ spec:
               name: settings-cms
             - mountPath: /openedx/config
               name: config
-            - mountPath: /openedx/uwsgi.ini
-              name: uwsgi-config
-              subPath: uwsgi.ini
+            - mountPath: /openedx/granian-config
+              name: granian-config
+              subPath: granian-config
           resources:
             requests:
               memory: 2Gi
@@ -113,12 +113,12 @@ spec:
         - name: config
           configMap:
             name: openedx-config
-        - name: uwsgi-config
+        - name: granian-config
           configMap:
-            name: openedx-uwsgi-config
+            name: openedx-granian-config
             items:
-              - key: uwsgi.ini
-                path: uwsgi.ini
+              - key: granian-config
+                path: granian-config
 ---
 apiVersion: apps/v1
 kind: Deployment
@@ -195,8 +195,8 @@ spec:
             value: lms
           - name: DJANGO_SETTINGS_MODULE
             value: lms.envs.tutor.production
-          - name: UWSGI_WORKERS
-            value: "{{ OPENEDX_LMS_UWSGI_WORKERS }}"
+          - name: GRANIAN_WORKERS
+            value: "{{ OPENEDX_LMS_GRANIAN_WORKERS }}"
           ports:
             - containerPort: 8000
           volumeMounts:
@@ -206,9 +206,9 @@ spec:
               name: settings-cms
             - mountPath: /openedx/config
               name: config
-            - mountPath: /openedx/uwsgi.ini
-              name: uwsgi-config
-              subPath: uwsgi.ini
+            - mountPath: /openedx/granian-config
+              name: granian-config
+              subPath: granian-config
           resources:
             requests:
               memory: 2Gi
@@ -224,12 +224,12 @@ spec:
         - name: config
           configMap:
             name: openedx-config
-        - name: uwsgi-config
+        - name: granian-config
           configMap:
-            name: openedx-uwsgi-config
+            name: openedx-granian-config
             items:
-            - key: uwsgi.ini
-              path: uwsgi.ini
+            - key: granian-config
+              path: granian-config
 ---
 apiVersion: apps/v1
 kind: Deployment

tutor/templates/kustomization.yml

@@ -62,9 +62,9 @@ configMapGenerator:
   options:
     labels:
         app.kubernetes.io/name: openedx
-- name: openedx-uwsgi-config
+- name: openedx-granian-config
   files:
-  - apps/openedx/uwsgi.ini
+  - apps/openedx/granian-config
   options:
     labels:
         app.kubernetes.io/name: openedx

tutor/templates/local/docker-compose.yml

@@ -97,13 +97,13 @@ services:
     environment:
       SERVICE_VARIANT: lms
       DJANGO_SETTINGS_MODULE: lms.envs.tutor.production
-      UWSGI_WORKERS: {{ OPENEDX_LMS_UWSGI_WORKERS }}
+      GRANIAN_WORKERS: {{ OPENEDX_LMS_GRANIAN_WORKERS }}
     restart: unless-stopped
     volumes:
       - ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
       - ../apps/openedx/settings/cms:/openedx/edx-platform/cms/envs/tutor:ro
       - ../apps/openedx/config:/openedx/config:ro
-      - ../apps/openedx/uwsgi.ini:/openedx/uwsgi.ini:ro
+      - ../apps/openedx/granian-config:/openedx/granian-config:ro
       - ../../data/lms:/openedx/data
       - ../../data/openedx-media:/openedx/media
       - ../../data/openedx-media-private:/openedx/media-private
@@ -124,13 +124,13 @@ services:
     environment:
       SERVICE_VARIANT: cms
       DJANGO_SETTINGS_MODULE: cms.envs.tutor.production
-      UWSGI_WORKERS: {{ OPENEDX_CMS_UWSGI_WORKERS }}
+      GRANIAN_WORKERS: {{ OPENEDX_CMS_GRANIAN_WORKERS }}
     restart: unless-stopped
     volumes:
       - ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
       - ../apps/openedx/settings/cms:/openedx/edx-platform/cms/envs/tutor:ro
       - ../apps/openedx/config:/openedx/config:ro
-      - ../apps/openedx/uwsgi.ini:/openedx/uwsgi.ini:ro
+      - ../apps/openedx/granian-config:/openedx/granian-config:ro
       - ../../data/cms:/openedx/data
       - ../../data/openedx-media:/openedx/media
       - ../../data/openedx-media-private:/openedx/media-private