openjph: HT-J2K encoder

[bradh]

This is an early-exposure draft of an OpenJPH based High Throughput JPEG-2000 encoder.

Thanks to @aous72 for OpenJPH and for debugging assistance. Also thanks to @Jamaika1 for suggesting this at #420.

An OpenJPH-based decoder is practical, but is not yet written. In the meantime, the OpenJPEG decoder will work.

I'm not sure about the additional codec type (heif_compression_HTJ2K). It seems like the JPEG community think that HT-J2K is really (a variation of) JPEG 2000, but if we want to pick a working encoder for the user, it seems like it would be better to have a different value.

All input appreciated.

Note: This will likely have merge conflicts with #1071, which I'll resolve after that gets merged.

[farindk]

Minor thing: maybe this case can be simply combined with the heif_compression_JPEG2000 above.

[bradh]

Will see if we need to add something extra for the HT case, and combine if not.

[bradh]

Have combined it.

[farindk]

Thank you, this is a great addition.
I also think that adding a new compression format enum heif_compression_HTJ2K makes sense, since (at least for the moment) not all J2K decoders will support HTJ2K and thus it would be difficult to choose the correct plugin.
It might still not be that easy, as, according to #1054, there is no indication in the container whether the coded stream actually uses HT-J2K.
Fortunately, OpenJPEG can decode both, and we can simply add HTJ2K into the list of supported formats for it, but we also might need some code in core libheif to detect whether the stream is HTJ2K.

[bradh]

It might still not be that easy, as, according to #1054, there is no indication in the container whether the coded stream actually uses HT-J2K.

It would help to not have to parse the CAP segment. I've flagged that on the HTJ2K discord channel to see if could be possible.

[farindk]

Another reason to have a separate heif_compression_HTJ2K might be that we currently cannot process any encoder parameters before the plugin is selected. And the plugin is only selected based on the compression format.
I.e. if all J2K encoders would support HTJ2K, a simple "-p htj2k=true" would work, but since OpenJPEG doesn't encode HTJ2K yet, there cannot be a "-p htj2k=true".

Second option: we would put the burden on the user when the codec is selected. He has to know that "-e OpenJPEG" is ordinary J2K und "-e OpenJPH" is HTJ2K.
Encoders that support both might have a "-p htj2k" parameters.

Third option: we require that all JPEG2000 plugins have an ht2k parameter option and if it is set to a value not supported by the plugin, it fails with an error. Sounds like a silly idea, but we can have similar cases for other codecs, e.g. a h265 codec that cannot encode monochrome images. (Currently we handle this by the plugin encoding dummy color planes.)

Fourth option: when selecting the encoder, we pass a compression-format specific options struct to libheif. For Jpeg2000, this would include the HTJ2K flag and would be used for plugin selection. However, this may contradict the explicit user choice through "-e".

My subjective summary: option 1 sounds like the easy solution. Maybe in the future, when all encoders support everything, the two enums can be synonyms.
My unexpected second place goes to options 3 and 4, because we might need a more flexible plugin selection anyways.

[bradh]

My unexpected second place goes to options 3 and 4, because we might need a more flexible plugin selection anyways.

Maybe this is worth deeper consideration. I did some investigation into hardware based encoding and decoding (e.g. with NVIDIA GPUs).

There are quite a few dimensions to whether the hardware can handle the task, beyond just the codec. NVIDIA documentation for the decoder and encoder shows differences in:

• bit depth
• supported profiles
• maximum size
• chroma

Running the tests code shows additional variation in MB count, minimum size and output surface format.

This is a tricky tradeoff, because we'd like to use the fastest codec that supports the configuration the user has, but that isn't really exposed in the plugin API.

[bradh]

I have cleaned up the code, and think it is now ready for review.

The one thing that doesn't work yet is the quality setting. That is still to be implemented in OpenJPH.

Also, there is no OpenJPH decoder implementation yet. I do plan to do that, no schedule at this stage.

[farindk]

Thank you.