Skip to content

perf: speed up distinct index collection in low-memory pipeline#711

Open
hjanuschka wants to merge 3 commits into
libjxl:mainfrom
hjanuschka:perf/pr05-distinct-indices-fastpath
Open

perf: speed up distinct index collection in low-memory pipeline#711
hjanuschka wants to merge 3 commits into
libjxl:mainfrom
hjanuschka:perf/pr05-distinct-indices-fastpath

Conversation

@hjanuschka

Copy link
Copy Markdown
Collaborator

This removes Option staging in get_distinct_indices and builds the result directly in position order. It reduces overhead in a hot helper used during low-memory pipeline row setup without changing behavior. Unsafe is used only to reborrow previously collected mutable pointers, and is safe because indices are distinct and sorted, so aliasing cannot occur.

@github-actions

github-actions Bot commented Mar 16, 2026

Copy link
Copy Markdown

Benchmark @ e39e2c5

MULTI-FILE BENCHMARK RESULTS (8 files)
  CPU architecture: x86_64
  WARNING: System appears noisy: high system load (2.03). Results may be unreliable.
Statistics:
  Confidence:               99.0%
  Max relative error:        3.0%

Comparing: 462454f5 (Base) vs cfa1595f (PR)

File Base (MP/s) PR (MP/s) Δ%
bicycles.jxl 6.975 6.982 +0.10% ±0.7%
bike.jxl 24.708 24.938 +0.93% ±1.7%
delta_palette.jxl 6.101 6.128 +0.43% ±0.3%
green_queen_modular_e3.jxl 8.259 8.194 -0.79% ±1.1%
green_queen_vardct_e3.jxl 25.305 25.187 -0.46% ±1.9%
lz77_flower.jxl 3.316 3.353 +1.13% ±0.3%
patches_lossless.jxl 3.178 3.176 -0.05% ±0.8%
sunset_logo.jxl 2.665 2.744 +2.95% ±1.1%

@veluca93 veluca93 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please (1) add an arbtest that checks that the code is fine (2) run this test under miri

@veluca93

Copy link
Copy Markdown
Member

This also needs a benchmark, enqueued it :-)

@veluca93 veluca93 force-pushed the perf/pr05-distinct-indices-fastpath branch 2 times, most recently from d3d0224 to b28f043 Compare June 1, 2026 16:51
@veluca93 veluca93 force-pushed the perf/pr05-distinct-indices-fastpath branch from b28f043 to d0d269c Compare June 9, 2026 22:22
@veluca93 veluca93 force-pushed the perf/pr05-distinct-indices-fastpath branch from d0d269c to dede62a Compare June 9, 2026 22:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants