Images with small changes end up bloating files

[spillerrec]

Some files contains a slight difference in large parts of the image of unknown reasons, for example:

Since we don't have access to the previous image, this is a lot less efficient than LZMA. An idea would be to store the difference between the two images, which I haven't had a lot of success with. However notice that most of the image is only a very slight change, this mask only reacts on differences above 12:

We could use the difference to store the small pixel value changes, and use the normal approach for the large changes. Like so:


This could perhaps easily split which pixels benefit from using a difference, and those which are better off stored normally. It seems to be the case, with this example saving of 60% of the file size, but this is just a quick example produced in Gimp which could contain errors!

This also raises the challenge of how to decide when to do it, as if we are combining and extracting frames, we have less control of which previous image it is doing the diff on. We should try making an implementation just for testing however, as this could result in significant savings for a certain set of images.

[spillerrec]

In a difference image, only pixels different than 0 will be changed, thus those pixels will not be affected of that conversion. We can use this to allow pixels to change, if we only consider pixels with a value different from 0 to affect the image.

[TsXor]

You may refer to opencv's source code for cv2.subtract because it can overcome jpeg artifacts.

[TsXor]

cv2 subtract
https://images1.tqwba.com/20201013/yny5szmoaqd.png
https://images1.tqwba.com/20201013/15i31gv322o.png
numpy minus
https://images1.tqwba.com/20201013/gsjrmoo3vrv.png
https://images1.tqwba.com/20201013/fz2akrzu41q.png

according to
https://www.tqwba.com/x_d/jishu/314357.html

[spillerrec]

@TsXor The difference comes from a lack of understanding arithmetic in computers. You receive the images as uint8 which can only represent the numbers [0, 255]. Any negative numbers wraps around and becomes positive, e.g. 3 - 5 = 254. What OpenCV does is to clamp the result so 3 - 5 = 0.

If you want to find what part of two images are different, you should use the absolute difference:

diff = abs(img1.astype(np.int16) - img2.astype(np.int16)).astype(np.uint8)

This will give the same result even if you swap img1 and img2 and not overlook some differences like cv2.subtract.