Skip to content

docs: clarify Redis Cluster compatibility#1161

Open
Yusufihsangorgel wants to merge 2 commits into
hibiken:masterfrom
Yusufihsangorgel:docs-redis-cluster
Open

docs: clarify Redis Cluster compatibility#1161
Yusufihsangorgel wants to merge 2 commits into
hibiken:masterfrom
Yusufihsangorgel:docs-redis-cluster

Conversation

@Yusufihsangorgel

Copy link
Copy Markdown

What this does

Replaces the single vague line under "Redis Cluster Compatibility" in the README

Some of the lua scripts in this library may not be compatible with Redis Cluster.

with a short section that explains how the hash-tag key design keeps a queue's
keys on one node, lists which features work on Redis Cluster, and spells out the
real operational caveats. It links to the existing Redis Cluster wiki page for the
walkthrough.

Closes #1086 (which scripts are incompatible and why).
Closes #1065 (list the features that work on Redis Cluster).

Why

The old sentence says "some scripts may not be compatible" without saying which,
so nobody can tell whether Redis Cluster is safe for their deployment. I went
through the code to answer that concretely.

What the README now says, and how I checked it

Every per-queue key is wrapped in a hash tag, asynq:{<queue>}:...
(internal/base/base.go), so all of a queue's keys hash to one slot and live on
one node. I went through all 50 redis.NewScript definitions in the library
(25 in internal/rdb/rdb.go, 24 in internal/rdb/inspect.go, 1 in
x/rate/semaphore.go) and, for each, listed every key it touches — including keys
built by string concatenation inside Lua and the unique_key read back out of a
task hash — and checked whether they all fall in the same slot.

They do. Each per-queue script only touches that queue's keys; the paired
server/worker records share a {host:pid:sid} tag; and the global sets
(asynq:queues, asynq:servers, asynq:workers, asynq:schedulers) are updated
by their own single-key commands rather than from inside a per-queue script. The
ListServers/ListWorkers/ListSchedulerEntries scripts touch only the single
registry ZSET and hand the member keys back to Go, which fetches each one with a
separate command. So none of the scripts issue a CROSSSLOT as the public API
calls them. RedisClusterClientOpt and the ClusterKeySlot/ClusterNodes
helpers already treat cluster as a supported mode.

The honest limitations are operational rather than "this script is broken": a
single queue is served by one node (you scale by adding queues), BatchEnqueue
across queues is a pipeline and so is not atomic across them, and operations can
briefly fail for a queue while its slot is being resharded. Those are what the new
section calls out.

How this was verified

Two layers. First, the static audit above: every script's key set, checked with
the slot arithmetic (all of one queue's keys map to one slot, each global set to
its own; the computation matches Redis's documented KEYSLOT("foo") == 12182).

Second, a runtime pass against a real 3-master Redis Cluster (Redis 8.4.0, no
replicas), using a small Go program built against this tree and connected with
RedisClusterClientOpt. It exercises every row of the matrix: immediate,
scheduled, unique, and batch enqueue; worker processing with forced failures
through retry into archive; completed-task retention; group aggregation; a
periodic task with scheduler history; the Inspector surface (stats with memory
sampling, per-state listings, run/archive/delete task, pause/unpause, cancel
processing, queue removal, cluster key slot and nodes); and the x/rate
semaphore. Queue names were picked so their slots land on three different
masters. Every returned error and every internal asynq log line was scanned for
CROSSSLOT/MOVED: zero occurrences, and each feature behaved the same as on a
single Redis instance.

Credit

Thanks to @luca-arch for the starting analysis in #1086 — pinning the scripts to
internal/rdb/rdb.go and internal/rdb/inspect.go and framing this around
CROSSSLOT. One refinement from going through the code: building a key inside Lua
(local key = ARGV[1] .. id) is not itself a cluster problem here, because those
keys keep the queue's {...} tag and stay in the same slot; dequeueCmd is the
clearest example. That's why the scripts hold up on a cluster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clarify which Lua scripts are incompatible with Redis Cluster Doc Improvement: List features work well on Redis Cluster

1 participant