Generic API for encoder options

[RunDevelopment]

The story for saving images with custom encoding options isn't great. Encoders are a patchwork of different API styles and capabilities, and the trait governing them has fundamental limitations too (#2556).

In this PR, I hope to improve the situation a little by adding an API for universal encoding options. Example usage:

let image = DynamicImage::new(32, 32, ColorType::Rgba8);

let mut options = image::codecs::png::PngOptions:.default();
options.compression = image::codecs::png::CompressionType::Best;
image.save_with_options("image.png", &options)?;

Instead of having to use encoders and their (arguably) low-level APIs directly, users can now provide a struct of encoding options instead. This makes it easy to configure encoders in a generic way.

Formats that currently support encoding options are PNG, JPEG, AVIF, PNM, and TGA.

Changes:

• Add EncodingOptions trait.
• Add {DynamicImage,ImageBuffer}::save_with_options and save_buffer_with_options.
• Add options structs for formats.

TODO:

• JPEG quality. The encoder doesn't have an option to change it. Can be resolved by either changing jpeg-encoder (Add quality getter and setter to Encoder vstroebel/jpeg-encoder#36) or changing our encoder. For now, I just excluded quality form the options that can be set.

---

Should help with #2810

[197g]

Defining types local to image, with some amount of dyn Any, is surely the way to go. Goals should be not exposing more of the underlying format than we can promise to maintain while making it possible to mostly reverse the decoding process. But since it's the most ambitious design extension and we have not validated any variant of this yet I want to be somewhat careful.

Have you considered an alternatives? The single downcast is quite strange, you'd need to match the type on both sides with no typing in between. Also, there's no general case. The standard Error type has a provide design. Would that be promising?

[RunDevelopment]

Thanks for taking a look.

I'm also not 100% happy with the design yet. I just started with the high-level API I wanted (image.save_with_options(path, &options)) and went from there. So if you have other ideas to get something similar, I'd love to hear them.

Regarding not exposing more than we can maintain: Options (at least right now) don't expose anything new. It's just the options encoders already have packaged differently.

Aside: Speaking of packaging, I'm also thinking about using this to unify the way encoders are configured. Right now, it's mess of API styles. Saying "all formats have an XEncoder and XOptions, use XEncoder.set_options" would be nice. This would obviously be breaking though, so maybe in another PR. It's orthogonal to a high-level options API anyway.

I also want to say that I wanted this options format to be extensible to third-party encoders à la #2916. I also wanted it to be extensible with other metadata too (e.g. XMP) but that didn't work out too well.

Have you considered an alternatives?

With the goal of providing a simple high-level API (image.save_with_options(path, &options)), I couldn't think of much better alternatives. I considered having all formats share a single options type but discarded the idea because formats are just too different from each other, and having most formats ignore most options in that type seemed suboptimal.

I also looked at other libraries.

• OpenCV's imwrite takes an array of ints, which is parsed as a list of key-value pairs. E.g. imwrite("test.png", img, [IMWRITE_PNG_COMPRESSION, 9, IMWRITE_PNG_FILTER, IMWRITE_PNG_FILTER_SUB]). Horrible API from a correctness perspective, but surprisingly easy to use and super FFI-friendly. It's completely untyped and all options are defined as constants (keys) with their possible values defined in their docs.
• Python's PIL uses format-specific overwrites. So e.g. image.save("name.jpeg", quality=90) and image.save("name.png", compress_level=9). Only works because these parameters are untyped.
• sharp as a generic method for saving (e.g. img.toFile("test.png")) and separate methods for each format it supports, e.g. img.jpeg({ quality: 80 }).
• C#'s ImageSharp has a general image.save("name.ext")API and one generic over encoders.

I don't think these APIs translate well for us. However, one commonality is that they all support a lot of options. Probably because it's useful to users.

The single downcast is quite strange, you'd need to match the type on both sides with no typing in between.

There is some typing with &dyn EncoderOptions (we're not quite &dyn Any) and only the lower-level APIs have to deal with casting. The idea was that higher-level APIs get the format from the options (ergo no mismatch possible). The level below that is using the encoder directly, which means unified options aren't necessary since you have direct access to the encoder.

We could also add something like XEncoder::set_options(&mut self, &XOptions) like I said above. This would be fully typed, but probably also not super useful right now.

The standard Error type has a provide design. Would that be promising?

Promising for what? We could attach bad options, but that's about it. Also nightly.

[197g]

I also looked at other libraries.

Very nice, quite a wide selection of styles that would each not fit Rust's idiomatic implementation well. Yes, I'd also very much avoid OpenCV. The PIL dismissal stands out though, the PR also has "untyped" parameters but they are not named.

While true that we can not name arguments in the call it is instead more feasible to use the type itself, which would be awkward in Python i.e. quality is a float and newtype wrapers are not idiomatic there. Also, the implication here of overloading everything at the top-level is that many options can be and are shared between multiple encoders, their name being convention as to meaning, while only some are in format-specific ones that require domain knowledge on the specific target. The flat hierarchy has some design consequences. qtables is an option for jpeg but absent from tiff's configuration even with jpeg compression selected, but the generic 'quality' parameter is used for several compression options.

---

I just started with the high-level API I wanted […]

I think the general shape is fine! The main design problem is that the lack of clear motivating example. The usage you're talking about works only in a monomorphic setting: the caller needs to know it is jpeg so that the jpeg's option struct can be constructed, yet the main mechanism is one of making the type information runtime to the library. That's a strange contrast. Would it not be nicer to have the JpegOptions construct the encoder, if such value must exist anyways at that point? That's one less import and the well-established builder pattern. I don't think the implementation is wrong but that motivation does not provide design validation.

---

Promising for what? We could attach bad options, but that's about it.

I'll preface this with a caution. Please don't read the below as inviting complexity. If it can't be done in a simple way, the current API is probably better :)

Also nightly.

Yes, we'd bring our own struct here that could maybe verify if some option type was used. There's nothing special or even unsafe about it.

As a solution to providing multiple options values of different types. I'd hoped that for a generic high-level option struct where mainly compression is defined roughly in terms of perception metric in the CQP sense that is used with much more depth in video (industry used of this was confirmed to me at RustWeek; there's special hardware with acceleration for choosing such parameters in video but we won't need that). Other shared parameters, such as "comment" blocks shared by multiple formats, could be similarly abstracted.

Now, the high-level API would be amenable to this. Just attempt a downcast into different types afterall. But maybe we could do a little better in terms of API if this were not an afterthought?

The decoder could then fetch those general options in addition to format specific ones. This would allow control over encoders even when the format choice is a runtime one. Then of course you could provide format specific options in addition. If the options were an owned parameter then Tiff could retain them and pass them on to the inner jpeg/webp encoder.

[RunDevelopment]

many options can be and are shared between multiple encoders, their name being convention as to meaning, while only some are in format-specific ones that require domain knowledge on the specific target.

Not really. As the user, you're supposed to know what format you're encoding with. Names are shared by necessity, not for semantics. E.g. quality=100 has different meanings for JPEG and AVIF.

Plus, all options require knowing the format. I.e. quality=100 isn't going to do anything if you encode a PNG.

Would it not be nicer to have the JpegOptions construct the encoder

I really like that. (I initially wanted to write that this would be impossible, because it requires GAT over types but those were stabilized a while ago. I'm even using return position impl trait in traits. The future is now!)

This also solves the problem that options couldn't move data into the encoder, since options had to be passed around by reference. But if the options construct the encoder, then we can pass around options by value. This also solved the JPEG quality issue.

I'm also thinking that this would make for a nice path toward a unified options API style for encoders. I'm imagining that all encoders would just have fn new(writer) -> Self and fn with_options(writer, options) -> Self constructors where new is implemented as just returning with_options(writer, Default::default()). But again, this is orthogonal and can be done in later PRs.