docs: clarify Redis Cluster compatibility#1161
Open
Yusufihsangorgel wants to merge 2 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Replaces the single vague line under "Redis Cluster Compatibility" in the README
with a short section that explains how the hash-tag key design keeps a queue's
keys on one node, lists which features work on Redis Cluster, and spells out the
real operational caveats. It links to the existing Redis Cluster wiki page for the
walkthrough.
Closes #1086 (which scripts are incompatible and why).
Closes #1065 (list the features that work on Redis Cluster).
Why
The old sentence says "some scripts may not be compatible" without saying which,
so nobody can tell whether Redis Cluster is safe for their deployment. I went
through the code to answer that concretely.
What the README now says, and how I checked it
Every per-queue key is wrapped in a hash tag,
asynq:{<queue>}:...(
internal/base/base.go), so all of a queue's keys hash to one slot and live onone node. I went through all 50
redis.NewScriptdefinitions in the library(25 in
internal/rdb/rdb.go, 24 ininternal/rdb/inspect.go, 1 inx/rate/semaphore.go) and, for each, listed every key it touches — including keysbuilt by string concatenation inside Lua and the
unique_keyread back out of atask hash — and checked whether they all fall in the same slot.
They do. Each per-queue script only touches that queue's keys; the paired
server/worker records share a
{host:pid:sid}tag; and the global sets(
asynq:queues,asynq:servers,asynq:workers,asynq:schedulers) are updatedby their own single-key commands rather than from inside a per-queue script. The
ListServers/ListWorkers/ListSchedulerEntriesscripts touch only the singleregistry ZSET and hand the member keys back to Go, which fetches each one with a
separate command. So none of the scripts issue a
CROSSSLOTas the public APIcalls them.
RedisClusterClientOptand theClusterKeySlot/ClusterNodeshelpers already treat cluster as a supported mode.
The honest limitations are operational rather than "this script is broken": a
single queue is served by one node (you scale by adding queues),
BatchEnqueueacross queues is a pipeline and so is not atomic across them, and operations can
briefly fail for a queue while its slot is being resharded. Those are what the new
section calls out.
How this was verified
Two layers. First, the static audit above: every script's key set, checked with
the slot arithmetic (all of one queue's keys map to one slot, each global set to
its own; the computation matches Redis's documented
KEYSLOT("foo") == 12182).Second, a runtime pass against a real 3-master Redis Cluster (Redis 8.4.0, no
replicas), using a small Go program built against this tree and connected with
RedisClusterClientOpt. It exercises every row of the matrix: immediate,scheduled, unique, and batch enqueue; worker processing with forced failures
through retry into archive; completed-task retention; group aggregation; a
periodic task with scheduler history; the Inspector surface (stats with memory
sampling, per-state listings, run/archive/delete task, pause/unpause, cancel
processing, queue removal, cluster key slot and nodes); and the
x/ratesemaphore. Queue names were picked so their slots land on three different
masters. Every returned error and every internal asynq log line was scanned for
CROSSSLOT/MOVED: zero occurrences, and each feature behaved the same as on asingle Redis instance.
Credit
Thanks to @luca-arch for the starting analysis in #1086 — pinning the scripts to
internal/rdb/rdb.goandinternal/rdb/inspect.goand framing this aroundCROSSSLOT. One refinement from going through the code: building a key inside Lua(
local key = ARGV[1] .. id) is not itself a cluster problem here, because thosekeys keep the queue's
{...}tag and stay in the same slot;dequeueCmdis theclearest example. That's why the scripts hold up on a cluster.