Skip to content

Update Redis Cluster compatibility docs#1169

Open
ranmori wants to merge 1 commit into
hibiken:masterfrom
ranmori:master
Open

Update Redis Cluster compatibility docs#1169
ranmori wants to merge 1 commit into
hibiken:masterfrom
ranmori:master

Conversation

@ranmori

@ranmori ranmori commented Jul 16, 2026

Copy link
Copy Markdown

`

Redis Cluster Compatibility — verified findings (v0.26.0)

I went through internal/rdb/rdb.go and internal/rdb/inspect.go directly rather than relying on the AI summary linked above, to confirm the pattern and scope it precisely.

1. Which scripts are affected, and why

Redis Cluster routes a script to a node based on the hash slot of the keys declared in the KEYS table passed to EVAL/EVALSHA. If a script touches an additional key that it builds itself by concatenating strings from ARGV (rather than receiving it via KEYS), that key is invisible to the cluster's routing/slot-check step. If that self-built key doesn't happen to hash to the same slot as the declared KEYS, Redis raises CROSSSLOT Keys in request don't hash to the same slot at execution time.

16 of the 43 Lua scripts in the codebase do this — they take an id/state/group name via ARGV and concatenate it directly onto a key prefix inside the script body:

-- pattern used in the affected scripts, e.g. archiveTaskCmd:
redis.call("LREM", ARGV[5] .. state, 0, id)
redis.call("ZREM", ARGV[6] .. group, id)

instead of the cluster-safe form, which is:

redis.call("LREM", KEYS[some_index], 0, id)

2. Affected functionality (public Inspector methods)

Script Public method(s)
getTaskInfoCmd Inspector.GetTaskInfo
currentStatsCmd, groupStatsCmd Inspector.CurrentStats, Inspector.GroupStats
runTaskCmd, runAllCmd, runAllAggregatingCmd Inspector.RunTask, RunAllScheduledTasks, RunAllRetryTasks, RunAllArchivedTasks, RunAllAggregatingTasks
archiveTaskCmd, archiveAllCmd, archiveAllAggregatingCmd, archiveAllPendingCmd Inspector.ArchiveTask, ArchiveAllScheduledTasks, ArchiveAllRetryTasks, ArchiveAllAggregatingTasks, ArchiveAllPendingTasks
deleteTaskCmd, deleteAllAggregatingCmd, deleteAllPendingCmd Inspector.DeleteTask, DeleteAllAggregatingTasks, DeleteAllPendingTasks
removeQueueCmd, removeQueueForceCmd Inspector.RemoveQueue
forwardCmd, deleteExpiredCompletedTasksCmd, deleteAggregationSetCmd internal scheduler/server maintenance loops (task forwarding, completed-task cleanup) — not directly user-called

Not affected: core enqueue/dequeue/done/retry paths (enqueueCmd, dequeueCmd, doneCmd, retryCmd, scheduleCmd, etc.) all pass their keys through KEYS, not ARGV concatenation. So basic task processing works fine on Redis Cluster — it's specifically the introspection and management surface (the Inspector API, and by extension the CLI/Web UI, which are built on top of it) that's at risk.

3. Practical impact

  • If you're only enqueueing and processing tasks, you're unaffected.
  • If you use Inspector methods, the CLI (asynq tool), or the Web UI to inspect, run, archive, or delete individual tasks/groups, those calls can fail with CROSSSLOT errors on a real multi-node cluster, depending on where the relevant keys land.
  • Whether you hit it in practice depends on your key distribution — it won't reproduce on redis-cli --cluster with one node, only multi-node.

Correction to the thread's existing (Copilot-derived) answer

I traced where those ARGV-concatenated key prefixes actually come from (base.TaskKeyPrefix, base.QueueKeyPrefix, base.GroupKeyPrefix in internal/base/base.go), and every one of them is built as:

func QueueKeyPrefix(qname string) string {
	return "asynq:{" + qname + "}:"
}

All asynq keys are already hash-tagged on {qname}. Redis Cluster only hashes the substring between { and } when computing a key's slot — so asynq:{q}:t:123 and asynq:{q}:archived land on the exact same slot regardless of what comes after the tag. In every affected script, the ARGV-built key and the KEYS-declared keys in the same call are constructed from the same qname, so they're always in the same hash tag and therefore the same slot.

Practical conclusion: the ARGV .. id pattern is a real style inconsistency (and the scripts would break if the hash-tag discipline in base.go were ever violated), but for the current codebase it does not appear to cause CROSSSLOT errors — the existing {qname} tagging already provides the safety net. I checked RemoveQueue, forward, and ArchiveTask specifically for any cross-queue key mixing and found none; every script's ARGV keys resolve to the same tag as its declared KEYS.

What I'd actually propose in the PR

  1. Correct the community answer above ("CROSSSLOT risk") with this analysis — it's a more useful answer to the three original questions than "these scripts are broken."
  2. Update the one-line README caveat to be specific instead of vague: name the invariant that makes it safe ({qname} hash-tagging) so future contributors don't accidentally break it by building a key from two different queues' data in one script, which would trigger CROSSSLOT.
  3. Optionally: a lint/test that asserts every ARGV-built key prefix used inside a Lua script matches the queue's hash tag — cheap insurance against regression.
`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant