Skip to content

image processing adaptor#1686

Draft
mtuchi wants to merge 23 commits into
mainfrom
image-fn
Draft

image processing adaptor#1686
mtuchi wants to merge 23 commits into
mainfrom
image-fn

Conversation

@mtuchi

@mtuchi mtuchi commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

New @openfn/language-image-utils adaptor providing image processing utilities for OpenFn job code.

Details

Using joe's implementation i was able to do the following

Five exported operations, each accepting a base64 string, data URL, or Buffer as input:
- resize(input, { width, height }) — resizes to target dimensions using jimp; outputs { buffer, width, height }
- compress(input, { maxBytes, minQuality }) — quality-loop compression to a byte target; outputs { buffer, size, quality }
- stripMetadata(input) — strips all EXIF metadata (including GPS) by re-encoding through Jimp; outputs { buffer }
- embedMetadata(input, { comment }) — embeds a string in the JPEG EXIF UserComment field; outputs { buffer }
- metadata(input) — reads image dimensions and orientation without modifying the image; outputs { width, height, orientation, size }

  • All image-returning functions (resize, compress, strip, annotate) accept { parseAs: 'base64' } to return a base64 string instead of a Buffer, similar to parseAs convention in language-common
  • Image processing functions use jimp package. The piexifjs package is used only for EXIF UserComment injection in annotate function

Here is sample job code

const url = 'https://tinyurl.com/rucpr6tk';

http.get(url, { parseAs: 'base64', maxRedirections: 1 });
resize($.data, { width: 1200, height: 1600 });
embedMetadata($.data.buffer, { UserComment: 'patient-id=42' });
compress($.data.buffer, { maxBytes: 700 * 1024 });

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to
know!):

  • I have used Claude Code
  • I have used another model
  • I have not used AI

You can read more details in our
Responsible AI Policy

Review Checklist

Before merging, the reviewer should check the following items:

  • Does the PR do what it claims to do?
  • If this is a new adaptor, added the adaptor on marketing website ?
  • If this PR includes breaking changes, do we need to update any jobs in
    production? Is it safe to release?
  • Are there any unit tests?
  • Is there a changeset associated with this PR? Should there be? Note that
    dev only changes don't need a changeset.
  • Have you ticked a box under AI Usage?

setup library for image processing and add few util functions
add unit test for process function
@mtuchi mtuchi changed the title add process function image processing adaptor Jun 3, 2026
@mtuchi

mtuchi commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator Author
image-fn.mp4

@mtuchi mtuchi requested a review from josephjclark June 3, 2026 16:23
Comment thread packages/image-fn/src/Adaptor.js Outdated
@mtuchi

mtuchi commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator Author

Hiya @josephjclark i have added processJsquash which uses @jsquash library.

jimp vs @jsquash/jpeg

jimp @jsquash
Output size (q80) ~389KB ~312KB (–20%)
Input formats JPEG, PNG, BMP, GIF, TIFF JPEG only
Setup cost None ~200ms WASM init
Compression loop 4170ms 2512ms (40% faster)

For this use case (single CommCare JPEG per job): differences are minor. @jsquash wins on compression efficiency; jimp wins on format flexibility.

⚠️ If we ever need to accept PNGs or screenshots, @jsquash throws at runtime

I think the right choice depends on use case. But i will pick jimp over @jsquash because of multiple image support and simple implementation. @jsquash wasmBinary feels a bit complicated to tbh

@josephjclark

Copy link
Copy Markdown
Collaborator

@mtuchi jimp is fine by me - let's optimise when we need to. I'm a little anxious about the wasm in the work (although it should be fine)

Please see my earlier comment about the design of the adaptor. Need to get that right before I can look closer

…ompress()

Drop wasm dependencies (@jsquash/jpeg, @jsquash/resize) in favour of
jimp-only pipeline. Split the combined process() into two composable
operations — resize() and compress() — each accepting an explicit base64
string or Buffer. Keep process() as a backward-compatible wrapper.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mtuchi

mtuchi commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator Author

Lets call it image-utils

mtuchi and others added 5 commits June 5, 2026 11:50
Rename adaptor to @openfn/language-image-utils. Drop the combined
process() function in favour of four focused utilities — strip() removes
EXIF metadata explicitly, metadata() exposes dimensions/orientation/size
for job-code validation, toBase64() and toBuffer() handle buffer/string
conversion and enable passing buffers between steps. Remove gif and avif
test fixtures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ction

Remove standalone toBase64/toBuffer operations. All image functions
(resize, compress, strip) now accept a base64: true option — when set
the output buffer key is replaced with a base64 string. Uses
buffer.toString('base64') directly since util.encode is text/JSON-
oriented and not suitable for binary image data.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace { base64: true } with { parseAs: 'base64' } on resize, compress,
and strip to match the parseAs convention used in language-common http.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add annotate(input, { comment }) for embedding EXIF UserComment data.
Remove the comment option from compress() — compression and metadata
writing are separate concerns.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mtuchi

mtuchi commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator Author

Hiya @josephjclark i have addressed you're early feedback and cleanup .avi and .gif images, i will wait for you're feedback before cleaning up more images

@mtuchi mtuchi requested a review from josephjclark June 5, 2026 10:07
Comment thread packages/image-utils/test/Adaptor.test.js
Comment thread packages/image-utils/test/Adaptor.test.js Outdated
Comment thread packages/image-utils/test/Adaptor.test.js Outdated
* @state {ImageState}
* @returns {Operation}
*/
export function resize(base64ImgOrBuffer, options = {}) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why resize(image, options) and not resize(image, width, height)? What do we gain with the options object?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@josephjclark in options we have parseAs option which is by default set to buffer.

But there are couple of other benefits like if we decide to add other options like,fit for managing fit image aspect ration, or quality to control image quality

Should we consider
resize(image, width, height, options)
OR
resize(image, dimensions ={width, height}, options= {parseAs})

Comment thread packages/image-utils/src/Adaptor.js Outdated
Comment thread packages/image-utils/src/Adaptor.js Outdated
Comment thread packages/image-utils/src/Adaptor.js Outdated
Comment thread packages/image-utils/src/Adaptor.js Outdated
Comment thread packages/image-utils/src/Adaptor.js Outdated
Comment thread packages/image-utils/src/Adaptor.js Outdated
@mtuchi

mtuchi commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator Author

Hiya @josephjclark i have added some integration test based on recent feedback from @jfeldman-openfn, See justin comment here. I would love you're feedback when you get back

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants