Skip to content

Add 'stat' command and per-key memory estimation to rdb-cli#103

Draft
moticless wants to merge 4 commits into
redis:mainfrom
moticless:handlers-stat
Draft

Add 'stat' command and per-key memory estimation to rdb-cli#103
moticless wants to merge 4 commits into
redis:mainfrom
moticless:handlers-stat

Conversation

@moticless

@moticless moticless commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds memory-oriented capabilities to rdb-cli, surfaced through its generated output:

  1. A new stat command that prints a human-formatted, estimated memory-statistics report.
  2. New per-key memory specifiers for print (%n encoding, %z estimated bytes, %g largest element), plus optional printf-style field widths.

Both share a single key-size estimator, so the per-key %z number and the stat aggregates are computed the same way.

stat — generated output

$ rdb-cli dump.rdb stat
rdb-cli dump.rdb stat
Statistics (Memory is estimated):
type             keys          items  items/key   volatile   expired       memory        avg    mem%
hash          1560278       81456780         52          0         0        6.8 G      4.6 K   74.8%
set           7651482       27952535          3          3         3        1.5 G      216 B   16.9%
list          6017862        7906342          1          0         0      719.9 M      125 B    7.7%
string         106018              -          -          2         2       63.5 M      628 B    0.7%
stream              1            948        948          0         0       22.3 K     22.3 K    0.0%
TOTAL        15335641      117316605                     5         5        9.1 G
volatile keys hold 2.0 K (0.0% of memory)

Top 10 keys by memory:
memory type        db        items   avg item      ttl  key
61.2 M set          0      1369515       46 B     none  App:AccessGroup:001
57.4 M set          0      1290404       46 B     none  App:EntityIndex:All
38.2 M hash         0       858814       46 B     none  App:EntityMetadata:002
...

Extending rdb-cli Usage:

FORMAT_OPTIONS ('stat'):
        -t, --top <N>      Show the top N keys by estimated memory
        -n, --now <SEC>    Reference Unix time for expiry evaluation (Default: now)

print — extended per-key memory output

$ rdb-cli dump.rdb print --key "db=%d,key=%k,type=%t,enc=%n,bytes=%z,largest=%g,count=%i"
db=0,key=mylist1,type=list,enc=quicklist,bytes=66,largest=2,count=1
db=0,key=lzf_compressed,type=string,enc=string,bytes=180,largest=118,count=0
...

New specifiers for --key:

Specifier Meaning
%n Encoding (e.g. quicklist, listpack, intset, skiplist)
%z Estimated in-memory size in bytes
%g Size in bytes of the largest element

Specifiers also accept an optional width, e.g. %-20k, %10z.

Implementation notes

  • New handlersCommon module holds the shared key-size accumulator, the in-memory size estimator, and file/lifecycle plumbing; both the print and stat handlers compose RdbxHandlersBase and register their own data callbacks over it.
  • The size estimate uses rough per-encoding overhead constants in the spirit of Redis MEMORY USAGE — calibrated so large keys land within ~25% of live MEMORY USAGE. It is explicitly an estimate.
  • Shared JSON/string escaping moved to extCommon (rdbxOutputPlainEscaping).
  • New public API: RDBX_createHandlersToStat(p, topN, nowSecs, outFilename).
  • Tests added in test/test_rdb_to_mem_stat.c.

Status

Draft — opening for early feedback on the output format and on the memory-estimation heuristic.

🤖 Generated with Claude Code

moticless and others added 4 commits June 23, 2026 14:39
Introduce a new `stat` output format that aggregates while parsing and,
at end-of-RDB, prints a human-formatted memory-statistics report
(estimated): a by-type table (keys, items, volatile, expired, memory,
avg, mem%) with a TOTAL row, followed by the top-N keys by estimated
memory. Memory use is bounded (per-type aggregates + the top-N buffer),
independent of the number of keys in the dump.

Extend the `print` format with per-key memory specifiers: %n (encoding),
%z (estimated in-memory size in bytes) and %g (largest element size).
Format specifiers now also accept an optional printf-style width
(e.g. %-20k, %10z).

Factor the shared key-size accumulation, estimator and file/lifecycle
plumbing into a new handlersCommon module reused by both the print and
stat handlers, and share the JSON/string escaping helper via extCommon.

New public API: RDBX_createHandlersToStat().

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant