gst-plugin-objectstore

A GStreamer plugin that streams HLS output to an object store — Azure Blob Storage, Amazon S3, Google Cloud Storage, or the local filesystem — modelled after awss3hlssink from gst-plugins-rs.

It provides a single element, objectstorehlssink, which is a GstBin wrapping hlssink3 (falling back to hlssink2). Instead of writing playlists and media segments to the local filesystem, it intercepts the get-playlist-stream / get-fragment-stream / delete-fragment signals and uploads/deletes each object in the configured store.

The backend is selected by the uri property and implemented with the object_store crate (s3://, gs://, azure://, file://, …).

Built to be compatible with gst-plugins-rs 0.15.2 (GStreamer Rust bindings 0.25 / GStreamer 1.28).

Building

cargo build --release

The compiled plugin is target/release/libgstobjectstore.dylib (.so on Linux). Point GStreamer at it with:

export GST_PLUGIN_PATH=$PWD/target/release
gst-inspect-1.0 objectstorehlssink

Properties

The store-type selects the backend and uri provides the location (its path is the base object-path prefix). The matching per-backend properties and the AWS_* / AZURE_* environment variables override or augment the configuration.

Property	Type	Description
store-type	string	Backend to use: azure, s3 or local (default local)
uri	string	Object store URI: location + base path (s3://bucket/prefix, azure://container/prefix, file:///path). Required.
azure-account	string	Azure Storage account name (azure:// backend)
azure-container	string	Azure container override (azure:// backend)
azure-access-key	string	Azure Storage account (shared) key (azure:// backend)
azure-endpoint	string	Custom Azure endpoint URI (Azurite, sovereign clouds)
s3-bucket	string	S3 bucket override (s3:// backend)
s3-region	string	S3 region (s3:// backend)
s3-access-key-id	string	S3 access key id (s3:// backend)
s3-secret-access-key	string	S3 secret access key (s3:// backend)
s3-session-token	string	S3 session token (s3:// backend)
s3-endpoint	string	Custom S3 endpoint URI (MinIO, other S3-compatible services)
request-timeout	uint64	Per-request timeout in ms (default 15000)
hlssink	element	The underlying HLS sink (read-only) — configure target-duration, playlist-length, max-files, … on it
stats	structure	Upload statistics (read-only)

Backends & credentials

• Azure (store-type=azure, uri=azure://container/prefix): authenticate with azure-account + azure-access-key, the AZURE_* environment variables, or ambient managed identity. azure-endpoint targets Azurite / sovereign clouds.
• Amazon S3 (store-type=s3, uri=s3://bucket/prefix): use the s3-* properties or the standard AWS_* environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, AWS_ENDPOINT, …).
• Local filesystem (store-type=local, uri=file:///path): no credentials.

Segment duration and playlist rotation are controlled by the inner hlssink3. Get it via the read-only hlssink property and set its properties directly (see the example below).

Examples

A runnable example lives in examples/hls_azure.rs. To test locally against the Azurite emulator:

# 1. Start Azurite
docker run -d -p 10000:10000 mcr.microsoft.com/azure-storage/azurite \
    azurite-blob --blobHost 0.0.0.0

# 2. Create the container (well-known Azurite dev credentials)
az storage container create --name hls --connection-string \
  "DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;"

# 3. Stream to it (defaults target the Azurite emulator)
cargo run --example hls_azure

gst-launch-1.0 against the various backends:

# Azure
gst-launch-1.0 -e \
  objectstorehlssink name=os store-type=azure uri=azure://hls/stream1 azure-account=MYACCOUNT azure-access-key=MYKEY \
  videotestsrc num-buffers=300 ! x264enc key-int-max=30 ! h264parse ! os.video

# Amazon S3 (AWS_* in the environment, or via s3-* properties)
gst-launch-1.0 -e \
  objectstorehlssink name=os store-type=s3 uri=s3://my-bucket/live/stream1 \
  videotestsrc num-buffers=300 ! x264enc key-int-max=30 ! h264parse ! os.video

# Local filesystem
gst-launch-1.0 -e \
  objectstorehlssink name=os store-type=local uri=file:///tmp/hls \
  videotestsrc num-buffers=300 ! x264enc key-int-max=30 ! h264parse ! os.video

The element exposes request pads video and audio, exactly like awss3hlssink.

Tests

# Unit + element tests (no network):
cargo test

# End-to-end test against Azurite (opt-in):
docker run -d -p 10000:10000 mcr.microsoft.com/azure-storage/azurite \
    azurite-blob --blobHost 0.0.0.0
AZURE_HLS_TEST=1 cargo test --test hls_azure_integration -- --nocapture

The integration test creates the container itself (with a Shared Key signed request, so no az CLI is needed), streams a short clip through objectstorehlssink with segment rotation enabled, and then verifies the uploaded blobs by reading them back with object_store. It targets the Azurite emulator by default and can be pointed at real Azure with the AZURE_ACCOUNT / AZURE_ACCESS_KEY / AZURE_CONTAINER / AZURE_ENDPOINT environment variables.

CI

The Azure integration GitHub Actions workflow runs the full suite (including the Azurite integration test) on push, pull request, and manual dispatch. The build and tests run inside an Alpine (musl) container with the official gst-plugins-rs-hlssink3 package (the same 0.15.2 release this plugin targets). No secrets are required — Azurite is started inside the job.

When building on Alpine/musl, two RUSTFLAGS are required (the workflow sets them):

RUSTFLAGS="-C target-feature=-crt-static -C link-arg=-Wl,-z,stack-size=2097152"

• target-feature=-crt-static links dynamically so the shared GStreamer/GLib libraries can be found (Rust's musl target is static by default).
• stack-size=2097152 raises musl's default thread stack (128 KiB) to 2 MiB; the GStreamer/x264 media threads overflow the small default and crash otherwise.

License

MPL-2.0. Derived from the awss3hlssink element of gst-plugins-rs (Copyright © 2022, Daily).