use /LZWDecode filter for GIF and matching TIFF images

frechefuchs commented

2023-10-02 11:45:03 +00:00

The GIF image format uses LZW compression, which is also a valid compression format for TIFF images. The PDF format provides an /LZWDecode filter that seems like a perfect match to embed GIF and certain TIFF images without transcoding image data.

A test with img2pdf 3.3.0 (Ubuntu 20.04) shows that GIF and LZW compressed TIFF images are always transcoded into /FlateDecode streams by img2pdf.

$ gm convert logo: GIF:- | img2pdf | grep -ia filter
    /Filter /FlateDecode
$ gm convert logo: -compress lzw TIFF:- | img2pdf | grep -ia filter
    /Filter /FlateDecode

Wouldn't it make sense to preserve LZW encoded image data in these cases?

There's one caveat, though. The PDF/A standard explicitly forbids /LZWDecode data streams. That is, when the --pdfa option is given, image data should be transcoded to use /FlateDecode streams, again.

As an example, a PDF file using the /LZWDecode filter can be created using GraphicsMagick like this:

$ gm convert logo: -compress lzw PDF:- | grep -ia filter
/Filter [ /LZWDecode ]

The GIF image format uses LZW compression, which is also a valid compression format for TIFF images. The PDF format provides an `/LZWDecode` filter that seems like a perfect match to embed GIF and certain TIFF images without transcoding image data. A test with img2pdf 3.3.0 (Ubuntu 20.04) shows that GIF and LZW compressed TIFF images are always transcoded into `/FlateDecode` streams by img2pdf. ``` $ gm convert logo: GIF:- | img2pdf | grep -ia filter /Filter /FlateDecode $ gm convert logo: -compress lzw TIFF:- | img2pdf | grep -ia filter /Filter /FlateDecode ``` Wouldn't it make sense to preserve LZW encoded image data in these cases? There's one caveat, though. The PDF/A standard explicitly forbids `/LZWDecode` data streams. That is, when the `--pdfa` option is given, image data should be transcoded to use `/FlateDecode` streams, again. As an example, a PDF file using the `/LZWDecode` filter can be created using GraphicsMagick like this: ``` $ gm convert logo: -compress lzw PDF:- | grep -ia filter /Filter [ /LZWDecode ] ```

👍 1

josch commented

2023-10-02 11:59:07 +00:00

Owner

Yes, that would make sense if it is possible.

Do you have very large GIF images were re-encoding them slows you down?

One problem is, that Pillow does not give me access to the compressed data and img2pdf would need to learn how to access and extract just the right bits from the input image.

Would you like to propose a patch?

Yes, that would make sense if it is possible. Do you have very large GIF images were re-encoding them slows you down? One problem is, that Pillow does not give me access to the compressed data and img2pdf would need to learn how to access and extract just the right bits from the input image. Would you like to propose a patch?

frechefuchs commented

2023-10-02 12:36:59 +00:00

Author

Well, I do not really have a use-case for /LZWDecode streams in PDF files. In fact, I'm actually trying to avoid those, because they're forbidden for PDF/A compliant files. I have stumbled across the issue while testing what error my PDF/A validation software (veraPDF) returns when presenting a file using LZWDecode filter. Knowing that img2pdf preserves image data as far as possible I fired-up img2pdf just to find me wondering veraPDF not triggering an alarm. Only after inspection, I found out img2pdf output uses /FlateDecode streams.

And then, I'm neither a Python nor C guy. Sorry, all I can contribute is an idea for enhancement.

Well, I do not really have a use-case for `/LZWDecode` streams in PDF files. In fact, I'm actually trying to avoid those, because they're forbidden for PDF/A compliant files. I have stumbled across the issue while testing what error my PDF/A validation software ([veraPDF](https://verapdf.org/)) returns when presenting a file using `LZWDecode` filter. Knowing that img2pdf preserves image data as far as possible I fired-up img2pdf just to find me wondering veraPDF not triggering an alarm. Only after inspection, I found out img2pdf output uses `/FlateDecode` streams. And then, I'm neither a Python nor C guy. Sorry, all I can contribute is an idea for enhancement.

frechefuchs commented

2023-10-02 13:59:40 +00:00

Author

The libtiff-tools package contains a tool tiff2pdf that seems to serve a similar purpose than img2pdf, but just for TIFF images. Though, it doesn't seem to preserve LZW compression.

However, it is able to preserve image data of some sub-formats without transcoding. From the man page:

If the input TIFF contains single strip CCITT G4 Fax compressed information, then that is written to the PDF file without transcoding, unless the options of no compression and no passthrough are set, -d and -n.

If the input TIFF contains JPEG or single strip Zip/Deflate compressed information, and they are configured, then that is written to the PDF file without transcoding, unless the options of no compression and no passthrough are set.

Maybe the code can be used as a reference.

The `libtiff-tools` package contains a tool tiff2pdf that seems to serve a similar purpose than img2pdf, but just for TIFF images. Though, it doesn't seem to preserve LZW compression. However, it is able to preserve image data of some sub-formats without transcoding. From the man page: > If the input TIFF contains single strip CCITT G4 Fax compressed information, then that is written to the PDF file without transcoding, unless the options of no compression and no passthrough are set, -d and -n. > > If the input TIFF contains JPEG or single strip Zip/Deflate compressed information, and they are configured, then that is written to the PDF file without transcoding, unless the options of no compression and no passthrough are set. Maybe the code can be used as a reference.

josch added the

enhancement

help wanted

labels 2023-10-02 14:40:50 +00:00

josch commented

2023-10-02 14:41:36 +00:00

Owner

Thank you! I've tagged this issue as "enhancement" and "help wanted", so if somebody finds some time to implement this I'd be happy to review patches. 😄

use /LZWDecode filter for GIF and matching TIFF images #174