Skip to content

Latest commit

 

History

425 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Zizq

Zizq (/zɪsk/) is a fast and durable job queue packed into a single native binary, built on an embedded LSM database — not on Redis, and not on your RDBMS.

The server runs as a straightforward HTTP/2 and HTTP/1.1 API, with easy-to-use clients currently implemented in Node.js, Ruby, Elixir and Rust (and others planned soon).

CI

Features

Zizq supports a growing number of features.

  • Single self-contained executable— no separate data store required
  • Durable and atomic job queues
  • Easy to use HTTP/1.1 and HTTP/2 API with JSON or MsgPack
  • Cross-language job enqueueing and execution
  • Unlimited named queues
  • FIFO ordered and granular priority jobs
  • Scheduled jobs
  • Configurable backoff/retry policies
  • Configurable job retention policies
  • Unique job support (deduplicated enqueues)
  • Cron (recurring jobs) scheduling
  • Batched job support (folded/merged enqueues)
  • APIs to manage the queue contents, including jq expression filters
  • Very high job throughput (over 100K jobs/sec on a Macbook Pro M4)
  • An insightful zizq top command

Resources

Getting started

Download a release compatible with your system and extract it. You should put the executable somewhere on your PATH but you can also just run it from the current directory.

curl -sLO https://github.com/zizq-labs/zizq/releases/download/v0.7.0/zizq-0.7.0-linux-x86_64.tar.gz
tar -xvzf zizq-0.7.0-linux-x86_64.tar.gz

Starting the server

Note

Full documentation for the zizq command is available on the website.

Zizq is a single binary organised into subcommands. The serve subcommand starts the server. This is the default when no other subcommand is specified.

$ ./zizq serve
Zizq 0.7.0
Listening on 127.0.0.1:8901 (admin)
Listening on 127.0.0.1:7890 (primary)

When the server starts, it creates a root directory in which it stores all queue data. By default this directory is {PWD}/zizq-root/ but can be explicitly specified by providing the --root-dir flag, or by specifying the $ZIZQ_ROOT environment variable. The server listens on 127.0.0.1 port 7890 unless otherwise specified by --host and --port, or $ZIZQ_HOST and $ZIZQ_PORT.

Run zizq serve --help to see a complete list of available options. The defaults should be good even for production use, provided the server is not set up to listen on a public IP address.

Communicating with the server

Tip

Read our Getting Started guide and the Zizq HTTP API Reference for more info on using the API directly.

With the server up and running we can make some requests to enqueue and perform some jobs. You would usually use a client library to do this, but the Zizq API is easy to use directly over HTTP too. Examples here use HTTPie for simplicity.

Let’s enqueue our first job!

http POST http://localhost:7890/jobs --raw '{
    "queue":"example",
    "type":"hello_world",
    "payload":{"greet":"Universe"}
}'
HTTP/1.1 201 Created
content-length: 163
content-type: application/json
date: Sat, 25 Apr 2026 11:56:45 GMT
{
    "attempts": 0,
    "duplicate": false,
    "id": "03g0ej3ybwh5uh1ap10xgufm3",
    "priority": 32768,
    "queue": "example",
    "ready_at": 1777118205503,
    "status": "ready",
    "type": "hello_world"
}

Now let's process that job with a little bash script loop and a sprinkle of jq just to help with the JSON in bash.

Note

This script does not exit until it is interrupted. The while loop receives lines of JSON from the server. Blank lines are heartbeats and are skipped.

http --stream GET http://localhost:7890/jobs/take | while IFS= read -r job; do
  [[ "$job" = "" ]] && continue

  id="$(echo "$job" | jq -r ".id")"
  type="$(echo "$job" | jq -r ".type")"

  echo "Processing job with id=${id} type=${type}..."
  echo "$job" | jq

  echo "Acknowledging completion..."
  http --quiet POST "http://localhost:7890/jobs/${id}/success" </dev/null

  echo "Done."
done
Processing job with id=03g0ej3ybwh5uh1ap10xgufm3 type=hello_world...
{
  "id": "03g0ej3ybwh5uh1ap10xgufm3",
  "type": "hello_world",
  "queue": "example",
  "priority": 32768,
  "status": "in_flight",
  "payload": {
    "greet": "Universe"
  },
  "ready_at": 1777118205503,
  "attempts": 0,
  "dequeued_at": 1777118245966
}
Acknowledging completion...
Done.

Client libraries

Note

Full documentation for our client libraries is available on the website.

Zizq provides official client libraries under the MIT license. The goal is to provide clients for a number of common languages. We have started with Node.js, Ruby, Elixir and Rust.

Each handles connection pooling, serialization and concurrency for you, and each ships a worker that runs jobs with a configurable level of concurrency.

Jobs can be enqueued from one language and processed by a worker in another.

Node.js — repo · docs

import { Client } from "@zizq-labs/zizq";

const client = new Client({ url: "http://127.0.0.1:7890" });

await client.enqueue({
  type: "send_email",
  queue: "emails",
  payload: { userId: 42, template: "welcome" },
});

Ruby — repo · docs

class SendEmailJob
  include Zizq::Job

  zizq_queue 'emails'

  def perform(user_id, template:)
    Mailer.deliver(User.find(user_id), template)
  end
end

Zizq.enqueue(SendEmailJob, 42, template: 'welcome')

Elixir — repo · docs

defmodule MyApp.SendEmail do
  use Zizq.JobKind, type: "send_email", queue: "emails"

  @impl Zizq.JobKind
  def perform(%{"user_id" => id}), do: MyApp.Mailer.deliver(id)
end

MyApp.SendEmail.new(%{"user_id" => 42})
|> Zizq.enqueue(MyApp.Zizq)

Rust — repo · docs

#[derive(Serialize, Deserialize, JobKind)]
#[zizq(name = "send_email", queue = "emails")]
struct SendEmail {
    user_id: u64,
}

client.enqueue(SendEmail { user_id: 42 }).await?;

Want a client for Zizq in a language not currently supported? leave a feature request or build your own easily.

Viewing live queue activity

Zizq includes a terminal-based live queue viewer, which provides real time insight into what the queue is currently doing, what the backlog looks like and how deep the queue is at different priority levels. The top subcommand provides this functionality.

$ ./zizq top

By default zizq top connects to 127.0.0.1:8901. Specify --url to connect to a different host.

Scroll up and down the list to visualise the backlog. The current position in the list is presented in the overall queue depth in real time. You can view the jobs that are currently in-flight, ready and scheduled for a later time.

zizq top

Support & Feedback

If you need help using Zizq, create an issue on the Zizq repo. Feedback is very welcome.

Releases

Packages

Used by

Contributors

Languages