Skip to content

fix: render compressedDepth images instead of a broken image icon - #1256

Open
HibridOPT wants to merge 3 commits into
lichtblick-suite:developfrom
HibridOPT:bugfix/compressed-depth-images
Open

fix: render compressedDepth images instead of a broken image icon#1256
HibridOPT wants to merge 3 commits into
lichtblick-suite:developfrom
HibridOPT:bugfix/compressed-depth-images

Conversation

@HibridOPT

@HibridOPT HibridOPT commented Jul 28, 2026

Copy link
Copy Markdown

User-Facing Changes

compressedDepth image topics now render in the Image and 3D panels, with the panel's existing depth color modes applying to them, instead of showing a broken image icon.

Description

Fixes #1241.

decodeCompressedImageToBitmap builds a media type by prefixing image/ to the message format, so a depth image became image/16UC1; compressedDepth png — which no browser can decode, hence the broken image icon.

Three commits:

  1. A minimal grayscale PNG decoder. compressed_depth_image_transport wraps depth images in a single-channel PNG. createImageBitmap is not usable here: the browser always hands back 8-bit RGBA samples, throwing away half the precision of 16-bit depth values. This decoder keeps the samples intact. It handles the subset of PNG that OpenCV's imencode produces for a single-channel Mat — non-interlaced, 8- or 16-bit grayscale — which is what every ROS depth publisher goes through, and rejects anything outside that with a specific message. Inflate uses the platform DecompressionStream, so no new dependency.
  2. Conversion to the raw image the payload encodes. A compressedDepth payload is a 12-byte ConfigHeader followed by the PNG, not a browser media type. Unpacking it into a raw image lets it render through the existing raw image path, so the panel's color mode settings apply exactly as they do for an uncompressed depth topic. 16UC1 carries depth directly in millimetres; 32FC1 has its inverse depth quantization undone with the parameters from the header, where a zero sample marks a pixel the publisher could not measure. The RVL codec is reported as unsupported rather than failing obscurely.
  3. Routing. Those messages now go to the compressedDepth conversion. The work runs in the existing image decoder worker, so the inflate and per-pixel conversion stay off the main thread.

Checklist

  • The web version was tested and it is running ok
  • The desktop version was tested and it is running ok
  • This change is covered by unit tests
  • Files constants.ts, types.ts and *.style.ts have been checked and relevant code snippets have been relocated

Summary by CodeRabbit

  • New Features

    • Added support for displaying ROS compressedDepth images in 3D views.
    • Supports 16-bit integer and 32-bit floating-point depth formats.
    • Compressed-depth decoding now runs off the main thread for smoother rendering.
  • Tests

    • Added coverage for depth conversion, grayscale PNG decoding, invalid payloads, and unsupported formats.

claude added 3 commits July 28, 2026 21:02
`compressed_depth_image_transport` wraps depth images in a single channel PNG,
which the app currently has no way to read. `createImageBitmap` is not usable
for it: the browser always hands back 8 bit RGBA samples, which would throw
away half the precision of the 16 bit depth values.

This decoder keeps the samples intact. It handles the subset of PNG that
OpenCV's `imencode` produces for a single channel `Mat` - non-interlaced, 8 or
16 bit grayscale - which is what every ROS depth publisher goes through, and
rejects anything outside that with a specific message. Inflate uses the
platform `DecompressionStream`, so no new dependency is needed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CvJLEJUpndwivJrV9yUY7G
A `compressedDepth` payload is a 12 byte `ConfigHeader` followed by the PNG,
not a browser media type. Unpacking it into a raw image lets it render through
the existing raw image path, so the panel's color mode settings apply exactly
as they do for an uncompressed depth topic.

`16UC1` carries depth directly in millimetres. `32FC1` has its inverse depth
quantization undone with the parameters from the header, where a zero sample
marks a pixel the publisher could not measure. The RVL codec is reported as
unsupported rather than failing obscurely.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CvJLEJUpndwivJrV9yUY7G
`decodeCompressedImageToBitmap` builds a media type by prefixing "image/" to
the message format, so a depth image became "image/16UC1; compressedDepth png",
which no browser can decode - hence the broken image icon in the Image and 3D
panels.

Those messages are now routed to the compressedDepth conversion and decoded
through the raw image path. The work runs in the existing image decoder worker
so the inflate and per-pixel conversion stay off the main thread.

Fixes lichtblick-suite#1241

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CvJLEJUpndwivJrV9yUY7G
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Changes

Compressed depth rendering

Layer / File(s) Summary
Grayscale PNG decoding and test fixtures
packages/suite-base/src/panels/ThreeDeeRender/renderables/Images/decodeGrayscalePng.ts, packages/suite-base/src/panels/ThreeDeeRender/renderables/Images/decodeGrayscalePng.test.ts, packages/suite-base/src/testing/builders/GrayscalePngBuilder.ts
Adds grayscale PNG inflation, validation, scanline unfiltering, and test image generation for 8-bit and 16-bit samples.
Compressed depth conversion
packages/suite-base/src/panels/ThreeDeeRender/renderables/Images/decodeCompressedDepth.ts, packages/suite-base/src/panels/ThreeDeeRender/renderables/Images/decodeCompressedDepth.test.ts
Parses compressedDepth formats, decodes PNG payloads with quantization headers, converts 16UC1 and 32FC1 data, and tests validation failures.
Worker and renderable integration
packages/suite-base/src/panels/ThreeDeeRender/renderables/Images/ImageRenderable.ts, packages/suite-base/src/panels/ThreeDeeRender/renderables/Images/WorkerImageDecoder.ts, packages/suite-base/src/panels/ThreeDeeRender/renderables/Images/WorkerImageDecoder.worker.ts
Routes recognized compressedDepth images through the worker decoder and existing image conversion path.

Estimated code review effort: 4 (Complex) | ~45 minutes

Suggested reviewers: luluiz

Sequence Diagram(s)

sequenceDiagram
  participant ImageRenderable
  participant WorkerImageDecoder
  participant WorkerImageDecoderWorker
  participant CompressedDepthDecoder
  ImageRenderable->>WorkerImageDecoder: decode compressedDepth image
  WorkerImageDecoder->>WorkerImageDecoderWorker: forward image, format, and settings
  WorkerImageDecoderWorker->>CompressedDepthDecoder: decode PNG payload to RawImage
  CompressedDepthDecoder-->>WorkerImageDecoderWorker: return RawImage
  WorkerImageDecoderWorker-->>ImageRenderable: return ImageData
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 30.77% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes address #1241 by decoding and rendering compressedDepth 16UC1 and 32FC1 images through the existing image pipeline.
Out of Scope Changes check ✅ Passed The added grayscale PNG decoder, depth conversion, worker plumbing, and tests are all directly supporting the compressedDepth fix.
Title check ✅ Passed The title clearly summarizes the main change: rendering compressedDepth images instead of showing a broken icon.
Description check ✅ Passed The description matches the template with User-Facing Changes, Description, and Checklist, and it includes the linked issue plus testing notes.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

Copy link
Copy Markdown

@gabriela-almeida-ctw gabriela-almeida-ctw added the needs-triage Pull request needs a triage from a human label Jul 29, 2026
@gabriela-almeida-ctw

Copy link
Copy Markdown
Contributor

Hi @HibridOPT

Please allow us a little time for a first human-review.

Our automated worflows were triggered and there are some lint issues that must be addressed.
You should not worry about the failing with npm audit. Please ensure your changes follow the guidelines from CONTRIBUTING.md.

Thanks for your contribution. 😸

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-triage Pull request needs a triage from a human

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot render compressedDepth images

3 participants